Warehousing checking method and warehousing system

Abstract

The invention relates to the technical field of intelligent warehouse management and Internet of Things perception, in particular to a warehouse checking method and a warehouse system. The method comprises the following steps: performing inventory demand path analysis on storage to obtain an inventory path scheme; performing inventory task issuing on the robot according to the inventory path scheme to obtain a carrying task instruction; carrying out inventory article grabbing processing according to the carrying task instruction to obtain static contact characteristic data; performing dynamic interaction and multi-mode signal acquisition based on the static contact characteristic data to obtain interactive sensing data; performing sensing signal separation on the interactive sensing data to obtain a mechanical parameter characteristic table and an acoustic characteristic spectrum; and performing article type identification matching on the mechanical parameter characteristic table and the acoustic characteristic spectrum to obtain an article type judgment result. According to the invention, the inventory function is seamlessly integrated into the daily operation process, so that the dimensionality and accuracy of the inventory information are improved, and the overall efficiency of warehousing operation is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent warehouse management and Internet of Things sensing technology, and in particular to a warehouse inventory method and a warehouse system. Background Art

[0002] Existing warehouse inventory counting technology mainly focuses on the external identification and location confirmation of items, and its ability to perceive the internal status of items is seriously insufficient. When the packaging of items is intact but the interior is damaged or deteriorated, the traditional inventory counting method cannot discover these hidden problems, resulting in significant differences between inventory information and actual conditions, which in turn affects the accuracy of downstream business decisions; for special materials such as liquids and particles, traditional inventory counting methods are difficult to determine their actual filling volume or content status without opening or intervention. The inventory of such materials usually relies on manual sampling or rough estimation, which is not only inefficient, but also prone to subjective errors, and cannot meet the needs of modern refined warehouse management; the traditional inventory counting process is separated from daily operations, and special time and manpower are required for inventory counting. This fragmented management model not only increases additional labor costs, but also interferes with the normal rhythm of warehouse operations and reduces overall operational efficiency. Especially for large warehouses, the complete inventory counting cycle can take several days or even weeks, making it difficult to ensure the real-time nature of inventory data.

[0003] In summary, existing technologies have problems such as the inability to perceive the internal state of objects, difficulty in accurately counting special materials, and the separation of inventory from daily operations, resulting in low efficiency, which need to be solved urgently. Summary of the invention

[0004] Based on this, it is necessary to provide a warehouse inventory method and a warehouse system to solve at least one of the above technical problems.

[0005] To achieve the above purpose, a warehouse inventory method includes the following steps:

[0006] Step S1: Analyze the inventory demand path of the warehouse to obtain the inventory path plan; issue the inventory task to the robot according to the inventory path plan to obtain the handling task instruction;

[0007] Step S2: performing inventory item grabbing processing according to the handling task instruction to obtain static contact characteristic data; performing dynamic interaction and multimodal signal collection based on the static contact characteristic data to obtain interactive perception data;

[0008] Step S3: Separate the perception signals from the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature map; perform item type recognition and matching on the mechanical parameter feature table and the acoustic feature map to obtain an item type determination result; perform an internal state assessment of the item based on the item type determination result to obtain an internal state assessment table; estimate the quantity / filling level and generate a perception report for the items identified as container types in the item type determination result according to the internal state assessment table to obtain a material perception feature report;

[0009] Step S4: Synchronously update the inventory information based on the material perception feature report to obtain a database update result; perform a business process trigger judgment based on the database update result to obtain a warehousing state update instruction.

[0010] The present invention realizes the automatic generation and orientation of inventory tasks by intelligently analyzing the storage inventory status and automatically screening the items to be inventoried according to preset rules. Combining the warehouse layout and the capabilities of robots, the system can calculate the optimal operation path and robot allocation plan, improving the overall efficiency of the inventory operation and the resource utilization rate. In particular, by finely configuring the sensing action parameters and sensor working modes according to the item characteristics, the pertinence and high quality of subsequent data collection are ensured, laying a foundation for perceiving the internal status of items through physical interaction and overcoming the limitations of traditional inventory task issuance that only focuses on location and identification. The perception of the internal status and physical characteristics of items is seamlessly integrated into the daily automated handling operations. Through basic operations such as the robot-controlled slow approach, initial contact detection, constant force maintenance, grasping, and lifting, and at the same time using a high-precision multi-modal sensor array for real-time and high-frequency data collection, the mechanical and acoustic responses of the item under different stress states are obtained. Further, by performing item-specific sensing actions (such as shaking and squeezing) and superimposing micro-disturbances, the key characteristic signals of the internal material state and structure of the item are excited. Precise time alignment and noise reduction processing of the multi-modal data ensure the synchronization and purity of the collected data, providing high-quality original input for subsequent accurate analysis of the hidden information of the item and significantly enhancing the dimension of the inventory data. Through in-depth intelligent analysis of the collected multi-modal interaction perception data, a non-invasive and high-dimensional assessment of the item type, internal status, and quantity / filling amount is achieved. The system extracts quantitative features reflecting the physical properties, internal structure stability, and content status of the item from the force / torque, vibration, and acoustic signals and compares them with the known item feature database, not only improving the accuracy of item identification but also detecting problems such as internal damage, component loosening, particle caking, or abnormal liquid sloshing under intact packaging. Especially for traditional materials such as liquids and particles that are difficult to accurately inventory, by analyzing their specific dynamic responses and acoustic fingerprints, it is assisted to estimate the filling amount or approximate quantity in the container, greatly improving the efficiency and fineness of the inventory of special materials. The finally generated perception report provides rich and accurate material status information for downstream inventory management and business decision-making. Deeply integrating the material perception feature report obtained by intelligent analysis with the warehouse inventory management system realizes the real-time and synchronous update of inventory information. The system can automatically compare the differences between the perception data and the existing inventory records, and according to the preset rules and the evaluated reliability, automatically or after manual review, correct the inventory quantity, update the internal status mark of the item, and the last inventory time. More importantly, based on the abnormal situations detected by perception (such as serious quantity discrepancies and abnormal internal status), the system can automatically trigger corresponding business processes, including generating a transfer order to move the abnormal item to the inspection area, creating an inventory review task that requires manual verification, and adjusting the strategy of the item in the outbound picking (such as prohibiting outbound or giving priority to outbound).This automated, rule-driven exception handling process significantly improves the warehouse's response speed and processing efficiency for exceptions, ensuring the accuracy of inventory data and the smooth progress of business processes.

[0011] Therefore, the present invention provides a warehousing inventory method. By integrating a multi-modal sensor system (force / torque, vibration, acoustics) on an automated handling robot, physical interaction response data of items is synchronously collected during daily handling operations. Using advanced signal processing and machine learning algorithms, internal state characteristics of items are extracted from this data to achieve non-invasive detection of problems such as internal damage and abnormal filling levels. This method seamlessly integrates the inventory function into the daily operation process, not only significantly enhancing the dimension and accuracy of inventory information, but also achieving the coordination of inventory and operations, greatly improving the overall efficiency of warehousing operations, and providing a new technical path for modern refined warehousing management.

[0012] Preferably, the present invention also provides a warehousing system for executing the above-mentioned warehousing inventory method. The warehousing system includes:

[0013] An inventory task assignment module for analyzing the inventory demand path of the warehouse to obtain an inventory path plan; issuing inventory tasks to the robot according to the inventory path plan to obtain handling task instructions;

[0014] An item interaction perception and acquisition module for performing inventory item grasping processing according to the handling task instructions to obtain static contact characteristic data; performing dynamic interaction and multi-modal signal acquisition based on the static contact characteristic data to obtain interaction perception data;

[0015] A material feature intelligent analysis module for separating perception signals from the interaction perception data to obtain a mechanical parameter feature table and an acoustic feature map; performing item type recognition and matching on the mechanical parameter feature table and the acoustic feature map to obtain an item type determination result; performing an internal state evaluation of the item based on the item type determination result to obtain an internal state evaluation table; estimating the quantity / filling level and generating a perception report for items identified as container types in the item type determination result according to the internal state evaluation table to obtain a material perception feature report;

[0016] An inventory status update and trigger module for synchronously updating the inventory information based on the material perception feature report to obtain a database update result; performing a business process trigger judgment based on the database update result to obtain a warehousing status update instruction.

[0017] This warehousing system integrates an inventory counting task assignment module, an item interaction perception and acquisition module, a material feature intelligent analysis module, and an inventory status update and trigger module, achieving the automation, intelligence, and high efficiency of the warehousing inventory counting method. The inventory counting task assignment module can intelligently generate and optimize inventory counting tasks based on the warehouse status and preset rules, automatically plan the robot path and configure perception actions, improving the efficiency and pertinence of task planning. The item interaction perception and acquisition module synchronously collects multi-modal data based on high-precision force / torque, vibration, and acoustic sensors during the robot's automated handling process, and uses controlled physical interaction to stimulate the internal feature response of items, achieving non-invasive perception of the internal state and physical properties of items, overcoming the limitation of traditional methods that can only rely on external identification. The material feature intelligent analysis module can deeply process and analyze the collected complex perception data, accurately identify item types, evaluate the internal state (such as detecting hidden damage, looseness, caking, etc.), and innovatively assist in estimating the quantity or filling amount of materials such as liquids and particles that are difficult to accurately count, greatly enriching the dimension of inventory counting information and improving data accuracy. The inventory status update and trigger module synchronizes the perception and analysis results to the inventory database in real time, ensuring that the inventory records are consistent with the actual state of the items, and can automatically trigger subsequent business processes such as warehouse transfer, recheck, and adjustment of the outbound strategy according to the perceived anomalies (such as quantity differences, status anomalies), improving the response speed of anomaly handling and the overall operation efficiency of the warehouse. In summary, this warehousing system seamlessly integrates the inventory counting function into daily operations, providing a more refined, accurate, and efficient inventory management method than traditional methods, significantly enhancing the automation level and information quality of warehousing management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the step flow of a warehousing inventory counting method.

[0019] The implementation, functional features, and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The technical method of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those skilled in the art within the scope of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor methods and / or microcontroller methods.

[0022] It should be understood that although terms such as "first" and "second" may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0023] In an embodiment of the present invention, with reference to Figure 1 as shown, it is a schematic diagram of the step flow of the warehousing inventory method of the present invention. In this example, the warehousing inventory method includes the following steps:

[0024] Step S1: Analyze the inventory demand path of the warehouse to obtain an inventory path plan; issue an inventory task to the robot according to the inventory path plan to obtain a handling task instruction;

[0025] In an embodiment of the present invention, the current inventory status is obtained through database query, and a list of items to be inventoried is filtered out according to preset inventory rules (such as based on time, storage location, inventory quantity, or exception mark). The system uses the warehouse layout diagram and the real-time position of the robot, and combines path planning algorithms (such as (Algorithm and approximation algorithm for the traveling salesman problem), calculate the optimal ordered path to visit the storage locations of the items to be inventoried, and form an inventory path plan. The system obtains the detailed physical attributes (weight, packaging, contents) of the items to be inventoried and the operation capabilities (load, gripper, sensor configuration) of the available robots, and uses a matching scoring mechanism and an allocation algorithm to determine the most suitable robot to perform the task and generate a robot allocation result. Based on the robot allocation result and the item characteristics (material, contents), the system queries the perception action library, matches the action templates, and calculates the accurate perception action parameters (such as shaking amplitude, frequency, extrusion pressure / deformation) according to the specific parameters of the item / robot. These calculated parameters are adjusted by the safety limits of the item and the robot to form a safety-constrained action plan, and the sensors (sampling rate, sensitivity) are configured accordingly to generate a sensor configuration plan. Finally, the system integrates and packages the inventory path plan, the robot allocation result, the safety-constrained action plan, and the sensor configuration plan to form a structured handling task instruction containing item identification, storage location, operation type, execution order, perception action parameters, and an exception handling plan, and issues it to the designated robot execution terminal through a secure communication channel.

[0026] Step S2: Perform grasping processing on the inventory items according to the handling task instruction to obtain static contact characteristic data; based on the static contact characteristic data, perform dynamic interaction and multi-modal signal acquisition to obtain interaction perception data;

[0027] In the embodiments of the present invention, after receiving a handling task instruction, the instruction content is parsed, and according to the specified item type and sensing action requirements therein, the working parameters (sampling frequency, sensitivity) of the on-board multi-modal sensors (force / torque sensor, vibration sensor, miniature microphone) are configured to complete the pre-activation of the sensors. The robot navigates to the target storage location area, uses the vision system to accurately position the item's pose, calculates the optimal grasping point, grasping posture, and approaching trajectory, and generates a grasping execution plan. The robot slowly approaches the item along the planned trajectory, and real-time monitors the force sensor data. When the force in the approaching direction is detected to exceed a preset threshold (e.g., 0.5 N), it is determined that initial contact has occurred, and the sensor and robot pose data at this moment are recorded as contact trigger data. The robot then switches to the force control mode, maintains a constant low contact force (e.g., 2 N) for a short time according to the contact parameters, and collects sensor data during this stage for multi-point micro-deformation testing to obtain static contact characteristic data reflecting the item's surface and initial mechanical characteristics. Then, the robot gradually increases the grasping force to the target value according to the grasping plan, and after confirming a stable grasp, lifts the item to the suspended state. During this process, force / torque data are continuously collected to obtain the grasping process characteristic curve and the lifting gravity response curve respectively. The system analyzes the lifting gravity response curve to evaluate the suspended stability of the item. On the basis of the stable suspension of the item, the robot performs the sensing actions (shaking, squeezing) specified in the safety constraint action plan and superimposes micro-disturbances, and the sensor array synchronously collects high-frequency response data to obtain disturbance action response data. The system performs high-precision time alignment on the collected multi-modal sensor array data to obtain a time-synchronized data stream, and integrates all the data in the dynamic stages (curves, stability data, disturbance responses, synchronized data). Finally, the system performs signal synchronization verification and noise reduction processing (digital filtering, spectral subtraction) on the integrated dynamic interaction response data to obtain a noise-reduced signal data packet, and encapsulates it with meta-information such as item identification and transmits it to the central processing system through a secure channel to form interactive sensing data.

[0028] Step S3: Separate the sensing signals from the interactive sensing data to obtain a mechanical parameter feature table and an acoustic feature map; perform item type recognition and matching on the mechanical parameter feature table and the acoustic feature map to obtain an item type determination result; perform an internal state assessment of the item based on the item type determination result to obtain an internal state assessment table; estimate the quantity / filling level and generate a sensing report for the items identified as container types in the item type determination result according to the internal state assessment table to obtain a material sensing feature report;

[0029] In the embodiments of the present invention, the force / torque, vibration, and acoustic signal data are separated, and modal-specific preprocessing (low-pass filtering, band-pass filtering, short-time Fourier transform) is performed to obtain a classified signal feature set. The system extracts parameters reflecting mechanical properties such as the mass, stiffness, elasticity, and damping of an item from the force / torque and vibration signals to generate a mechanical parameter feature table. The system further performs time-frequency conversion (STFT) on the acoustic and vibration signals to generate a time-frequency energy matrix. Based on the time-frequency energy matrix, the system calculates the energy statistical characteristics of different material characteristic frequency bands and extracts the spectral shape parameters (such as spectral centroid, spread) of each time frame. At the same time, the system analyzes the energy envelope of the acoustic signal and extracts its dynamic parameters (duration, decay time). The system integrates these frequency band statistics, spectral shape, and envelope dynamic characteristics to form a high-dimensional acoustic feature vector, and performs pattern matching (such as Euclidean distance, similarity calculation) with a preset acoustic fingerprint database to obtain a material acoustic feature matching table and preliminarily identify the item type. Based on the identified container item type and acoustic characteristics (especially sloshing energy, decay time, particle energy, etc.), the system applies a pre-calibrated mapping model or regression formula to estimate the filling rate of the liquid or the quantity of particulate or solid items, and performs auxiliary verification in combination with mechanical parameters (such as estimated mass, moment of inertia). The system calculates the reliability score of the estimation result (based on signal-to-noise ratio, recognition confidence, modal consistency, etc.) and calibrates it using historical data to generate a filling quantity estimation result including the estimated value, error range, and reliability score. At the same time, the system compares the current time-frequency energy matrix with the normal item template to detect abnormal resonances, acoustic deviations, or transient noises, and generates an abnormal acoustic feature table. Finally, the system integrates the item type determination result, internal state assessment (judgment based on abnormal acoustic / mechanical characteristics), and filling quantity estimation result into a structured material perception feature report to comprehensively describe the perception state of the item.

[0030] Step S4: Synchronously update the inventory information of the material perception feature report to obtain a database update result; based on the database update result, perform a business process trigger judgment to obtain a warehousing status update instruction;

[0031] In the embodiments of the present invention, format, field, and identifier validity verification are first performed to generate a verification result record. For reports that pass the verification, the system queries the inventory master table to obtain the current item record, and carefully compares the item type, quantity / fill level, and internal status in the perception report with the inventory record, calculates the difference value and percentage, marks the status change, and generates a data difference analysis table. Based on the data difference analysis table and preset update rules (for example, update the quantity if the quantity difference exceeds the threshold, and update the status if the status becomes abnormal), the system generates database operation instructions (such as SQL UPDATE / INSERT statements) for modifying the inventory database, forming an update data packet. The system evaluates the permission requirements (automatic execution or manual review) and business priorities (high, medium, low) of each instruction in the update data packet according to the update policy matrix, obtaining a hierarchical update plan. The system filters out the instructions marked for automatic execution, starts a database transaction, executes these instructions in the order of priority, and records the transaction execution result (success / failure) and detailed logs, forming a database update result. The system parses the database update result, identifies the item records that actually changed and their change types (quantity change, status change, type change), and generates a classified update record table. The system matches the subsequent business process actions that need to be triggered (such as inventory transfer, recheck, adjustment of outbound strategy) according to the information in the classified update record table against the exception handling rule library, obtaining an exception handling plan table. Based on the exception handling plan table, the system generates a specific item inventory transfer instruction set (specifying the target storage location), an inventory recheck task table (specifying the recheck content and method), and an outbound rule modification instruction (specifying the new outbound status or priority). Finally, the system integrates these business process instructions (inventory transfer, recheck, outbound adjustment) with the database update result to form a complete warehousing status update instruction containing the details of inventory changes and subsequent action instructions, and sends it to the relevant warehousing execution and management systems.

[0032] Preferably, step S1 includes:

[0033] Step S11: Obtain the inventory status data of the current warehouse and generate an inventory check demand list according to the preset inventory check rules;

[0034] Step S12: Plan the operation path of the robot according to the inventory check demand list to obtain an inventory check path plan;

[0035] Step S13: Obtain the robot capability data and item characteristic data, and perform robot capability matching according to the inventory check path plan to obtain a robot allocation result;

[0036] Step S14: Configure the sensing actions of the robot according to the robot allocation result to obtain a sensing action configuration table;

[0037] Step S15: Encapsulate and issue instructions for the inventory path plan, robot allocation result, and perception action configuration table to obtain a handling task instruction.

[0038] In the embodiment of the present invention, by calling a database interface (for example, using an SQL query statement), all item records stored in the current warehouse are obtained from the backend inventory main table (Inventory_Main) and inventory history table (Inventory_History). The inventory main table at least includes the following fields: item identifier (Item_ID), current storage location (Location_ID), current inventory quantity (Current_Qty), inbound time (Inbound_Time). The inventory history table includes: item identifier (Item_ID), inventory time (Inventory_Time), inventory quantity (Inventory_Qty), inventory result status (Status). After the system receives these original inventory status data, they are loaded into memory for processing. The system internally stores a preset set of inventory rules, and this set defines the conditions for triggering an inventory task. For example, the rule set includes: Rule ①: The last inventory time of the item exceeds 90 days from the current time; Rule ②: The current storage location of the item belongs to the specified storage area range for the planned area inventory this time; Rule ③: The difference between the current inventory quantity of the item and the safety stock threshold (Safety_Stock) is less than -10% (indicating a serious shortage of inventory); Rule ④: Items detected with quantity or status abnormalities and marked by the system during other operations (such as outbound verification). The system traverses each item inventory record obtained and matches them one by one with the rule set. If a certain record meets any rule, the unique identifier (Item_ID) of the item and its current storage location information (Location_ID) are extracted and added to a temporary list. After traversing all the records, this temporary list is sorted out and named the inventory requirement list, which is an ordered or unordered set containing the identifiers of items to be inventoried and storage location information.

[0039] By querying the basic warehouse data, the system converts each storage location identifier into its corresponding three-dimensional space coordinates ( , , ). The system also obtains the real-time position information of the currently available robots ( , , ) and the global map data of the warehouse. The global map data is represented in a graph structure, where nodes represent channel intersections or storage locations, and edges represent channels, including the weights of the edges (such as walking distance, passing time). The operation path planning module uses the A algorithm (or other path search algorithms, such as the Dijkstra algorithm), with the current position of the robot ( , , ), starting from all the bin coordinates {( , , ),( , , ),...,( , , )} in the inventory requirement list as the set of mandatory points to pass through, and taking the position (x_charge, y_charge, z_charge) where the robot returns to the charging station as the end point, calculate a path sequence with the lowest total path cost (for example, the shortest total walking distance). To optimize the access order, an approximation algorithm for the Traveling Salesman Problem (TSP) can be combined outside the A algorithm to determine the best order to visit {( , , ),...,( , , )} these points. What the algorithm outputs is an ordered bin access sequence, for example: Loc_A1 → Loc_C7 → Loc_B3 →... → Loc_Charge. This ordered bin access sequence and the path for the robot to reach the first bin in the sequence from the current position together constitute the inventory taking path plan.

[0040] For each item identifier in the plan, the system retrieves the detailed physical properties of the item by querying the item property database. For example: the nominal weight of the item (Weight_Nominal, unit: kg), the packaging type (such as: carton, plastic drum, metal can), whether it contains special media marks such as liquid, particles or fragile components. At the same time, the system queries the capability list of all available robots in the current system. The robot capability list contains the unique identifier of each robot (Robot_ID), and the operation capability parameters of the robot, such as: the maximum grasping load (Max_Load_Capacity, unit: kg), the gripper type (Gripper_Type, such as: jaw, fork arm, suction cup), the type of integrated sensors (such as: force sensor, vibration sensor, microphone), and the sensor accuracy and sampling rate. The system evaluates the matching degree between the physical properties of each item to be inventoried in the inventory path plan and the available robot capability parameters. The matching degree evaluation uses a scoring mechanism, for example, constructing a matching function Score(Item_Properties,Robot_Capabilities). This function considers multiple dimensions: if the item weight Weight_Nominal > the robot Max_Load_Capacity, the matching degree is 0; if the item packaging type is incompatible with the robot Gripper_Type, the matching degree is 0; if the item contains liquid media and the robot is not equipped with vibration sensors and microphones, the matching degree is low; if the item requires high-precision force sensing and the accuracy of the robot force sensor is insufficient, the matching degree decreases. The system traverses all combinations of items to be inventoried and all available robots, calculates the matching degree score, and forms a two-dimensional matching matrix M, where M ij represents the matching degree between the i-th item and the j-th robot. The system adopts an allocation algorithm based on the matching matrix M (for example, the cost minimization allocation algorithm). On the premise of meeting the hard constraints (such as the load and gripper type must be compatible), it selects an optimal robot-item allocation plan to maximize the total matching degree score or minimize the total operation cost. This allocation plan clarifies which items or storage locations each robot is responsible for inventorying, and this plan is the robot allocation result.

[0041] Query the material information of the item or the preset material type according to each item identifier. There is a perception action rule library stored internally in the system. This rule library is a lookup table, where the key is the material type of the item or a combination of specific attributes (e.g., liquid container), and the value is the type of perception action recommended to be performed on the item of this type and the detailed parameters. For example, the rule library entries are: Rule A: Material type = liquid; Recommended action = shake; Parameters = {'amplitude': 5 degrees, 'frequency': 2 Hz, 'duration': 1 second}; Rule B: Material type = granular; Recommended action = vibrate; Parameters = {'type':'sweeping frequency', 'initial frequency': 50 Hz, 'end frequency': 150 Hz, 'duration': 1.5 seconds}; Rule C: Material type = solid in cardboard box; Recommended action = squeeze; Parameters = {'maximum pressure': 10 N, 'duration': 0.5 second, 'deformation limit': 5 mm}. Traverse each item in the robot allocation result. For each item, the system looks up the perception action rule library according to its material type and extracts the corresponding perception action type and parameters. If there is no directly matching rule in the rule library for a certain item type, or the responsible robot does not have the sensors required to perform the recommended action (e.g., the rule recommends shaking, but the robot does not have a force / torque sensor), the system will fallback to the default perception action (e.g., only perform basic grasping and lifting without performing additional micro-disturbances) or mark the item as unable to perform in-depth perception. The system records each item identifier, the corresponding perception action type and detailed parameters, forming a list. This list is the perception action configuration table, which specifies the perceptive interaction operations that the robot should perform for each item to be inventoried.

[0042] The inventory path plan provides the order of storage locations that the robot needs to visit. The robot allocation result determines which robot is responsible for visiting which storage locations. The sensing action configuration table specifies the sensing actions that need to be performed when visiting a specific storage location and grasping an item. The system constructs a structured data packet, which is the final handling task instruction sent to the robot. The instruction data packet includes: ① Task unique identifier (Task_ID): For example, generated based on a timestamp and a sequence number, such as "INV_20231027_001". ② Executing robot identifier (Robot_ID): Specifies the robot that receives and executes this task. ③ Operation type (Operation_Type): Specifies that this task is an "inventory operation". ④ Operation sequence (Operation_Sequence): This is an ordered list, and each element in the list corresponds to a storage location access step in the path plan. Each element includes: Target storage location identifier (Location_ID). Item identifier to be processed (Item_ID), including batch information. Detailed sensing action instruction (Sensing_Action_Instruction): Includes the action type (such as'shaking', 'extruding') and its specific execution parameters (such as amplitude, frequency, duration, pressure limit, etc.), which are extracted from the sensing action configuration table. Target placement location (Target_Location): Usually the original storage location, but if the sensing analysis determines that the item is abnormal, it needs to be moved to a specified inspection area. Exception handling plan (Exception_Handling): For example, the fallback strategy in case of grasping failure or sensor failure. The above structured data packet is transmitted through the warehouse network to the on-board control system of the specified executing robot (determined by Robot_ID) using a standard robot communication protocol (such as MQTT, ROS Action, or a custom TCP / IP protocol), securely encrypted. This transmitted data packet is the handling task instruction.

[0043] Preferably, the handling of grasping the in-stock items in step S2 includes:

[0044] Perform instruction parsing and sensor pre-activation on the handling task instruction to obtain a sensor configuration plan;

[0045] Based on the sensor configuration plan and the storage location information in the handling task instruction, perform proximity and positioning preparation for the items at the storage location to obtain a grasping execution plan;

[0046] Perform initial contact and obtain basic characteristic data according to the grasping execution plan to obtain static contact characteristic data.

[0047] In the embodiments of the present invention, after receiving a handling task instruction, the instruction is first parsed to identify the identifier of the item to be processed, the storage location information, and the types and parameters of the sensing actions to be performed. Based on this information, the system queries the internal sensor configuration rule library to determine which sensors need to be activated (such as force / torque sensors, vibration sensors, microphones), and sets appropriate operating modes for them, including sampling frequencies (for example, 150 Hz for the force sensor, 500 Hz for the vibration sensor, 16 kHz for the microphone), sensitivities (for example, the force sensor is set to the high-sensitivity mode of ±0.01 N), and data transmission formats, completing the pre-activation of the sensors and obtaining a sensor configuration plan.

[0048] Subsequently, the robot moves to the target storage location area using its navigation system according to the sensing requirements determined in the sensor configuration plan and the storage location information in the handling task instruction. After arriving at the storage location, the robot uses its on-board vision system (such as a binocular camera or an RGB-D camera) to collect images of the items in the storage location. The image processing module uses object recognition algorithms (such as the YOLOv7 or MaskR-CNN models based on deep learning) and pose estimation algorithms to locate the target item in the image and calculate the precise three-dimensional position (x, y, z) and pose (roll angle, pitch angle, yaw angle) of the item relative to the robot's base coordinate system. The grasping planning module uses this precise pose information, the known size model of the item, and the geometric model of the robot's end effector (gripper) to calculate the optimal grasping point and grasping pose, and at the same time determines the pre-grasping position and approaching speed of the robot arm before reaching the grasping point from the current position (for example, the approaching speed is set to 0.1 m / s). These calculation results, including the precise pose of the item, the coordinates of the optimal grasping point, the grasping pose, the approaching trajectory, the preset initial contact force threshold (for example, 0.5 N), and the grasping force parameters, together constitute the grasping execution plan.

[0049] The robot controls the movement of its arm according to the grasping execution plan, causing the gripper to slowly approach the object. During this process, the force / torque sensor monitors the force exerted on the gripper in real time. When the force detected by the sensor exceeds a preset initial contact force threshold in a certain axial direction (for example, the force Fz along the approaching direction of the gripper > 0.5 N), the robot motion control system determines that contact has been made with the object, immediately stops the main approaching motion, and switches to force control mode or impedance control mode to maintain a gentle and stable contact force (for example, keeping the force Fz along the contact direction within the range of 2 N ± 0.5 N) for a short period of time (for example, 1 second). During this stable contact period, all activated sensors (force / torque sensors, vibration sensors, microphones) collect data synchronously at a high frequency (determined by the sensor configuration scheme). The force / torque sensor records the reaction force change curve of the object under the stable contact force, reflecting the compliance and initial mechanical impedance of the object surface. The vibration sensor and microphone collect the transient vibrations and acoustic signals generated during contact, as well as the baseline response of environmental noise transmitted through the object. These original data streams of sensors collected during the stable low-force contact stage, which reflect the static or quasi-static physical responses of the object, are the static contact characteristic data after being timestamped and preliminarily formatted.

[0050] Of particular importance, the initial contact and basic characteristic acquisition include:

[0051] Determine the contact parameters according to the grasping execution plan;

[0052] Perform slow approach control according to the contact parameters to obtain the approach process record;

[0053] Perform initial contact detection according to the approach process record to obtain contact trigger data;

[0054] Perform constant force holding control according to the contact trigger data to obtain constant force control data;

[0055] Perform multi-point micro-deformation testing according to the constant force control data to obtain static contact characteristic data;

[0056] In the embodiment of the present invention, the grasping execution plan is received. This plan is calculated and generated by the upper-level system according to the object pose, type, and robot capabilities, and includes the preparatory pose of the robot end effector before reaching the object grasping point, the approaching trajectory vector from the preparatory pose to the grasping point (for example, a unit vector u representing the negative direction of the Z axis of the robot end effector coordinate system), and the relevant parameters of the expected initial grasping operation. The system extracts the control parameters for the initial contact stage from the grasping execution plan. These parameters include: the initial contact detection threshold (for example, set as the force along the approaching trajectory vector > 0.5 N), where is the dot product F·u of the force vector F measured by the robot end - effector force sensor and the approach trajectory vector u; the target constant contact force for the stable contact phase (e.g., set to a force along the direction of the approach trajectory vector = 2N); the time duration for which the stable contact force is maintained (e.g., set to 1 second). These extracted and set values together constitute the contact parameters. The robot first moves to a preparatory pose. Then, the robot control system executes a controlled linear motion command to slowly move the end - effector towards the item along the direction indicated by the approach trajectory vector u. In this stage, a position control mode is adopted, but a very low speed limit is set (e.g., the maximum movement speed is set to 0.05 m / s). While the robot is performing the slow approach, the on - board multi - modal sensors of the robot (force / torque sensor, vibration sensor, miniature microphone) continuously collect data at high frequencies (e.g., the force sensor at 1000 Hz, the vibration sensor at 500 Hz, the microphone at 16 kHz). The robot control system also records the precise position (x, y, z coordinates) and orientation (roll angle, pitch angle, yaw angle) of the end - effector over time. All this sensor data and robot state data, along with precise timestamps, are continuously recorded to form the approach process record. The system continuously calculates the component of the currently measured end - effector force vector F in the direction of the approach trajectory vector = F·u. The system compares the calculated value with the initial contact detection threshold (e.g., 0.5 N) set in the contact parameters. When it is detected that the value jumps from below the threshold and remains above the threshold (e.g., for 10 consecutive sampling points When the force sensed by the force / torque sensor exceeds a certain threshold (e.g., > 0.5 N), the system determines that the end effector of the robot has made effective contact with the surface of the object. At this time, the system immediately marks the current precise timestamp and records the original readings of all sensors (force / torque, vibration, microphone) and the pose information of the end effector of the robot at the triggering moment. This set of data marked with the triggering time point and the corresponding sensor / robot status information is the contact trigger data. The system then switches the control mode of the end effector of the robot from position control to force control along the approaching trajectory vector u (or other forms of impedance control or hybrid force / position control). The control objective is to make the end effector of the robot apply and maintain a constant force along the direction of u, and its magnitude is equal to the target constant contact force set in the contact parameters (e.g., 2 N). The control loop continuously receives the real-time force data feedback from the force / torque sensor and adjusts the torque of the robot joints or the position of the end effector according to the error between the target force and the actually measured force to reduce the error, thereby achieving a constant force output. During the set duration (e.g., 1 second) of the constant force maintenance control execution, all activated sensors (force / torque sensor, vibration sensor, miniature microphone) continue to collect data synchronously at a high frequency. These original sensor data streams collected under the constant contact force, together with the pose adjustment records of the end effector of the robot, are continuously recorded to form constant force control data. Receive the constant force control data, which includes the force / torque, vibration, and acoustic sensor data collected under the constant contact force and the robot pose records. During this constant force maintenance stage, the robot control system can also superimpose and execute preset micro-perturbation instructions. These micro-perturbations are small, controlled displacements or force changes applied to the object without significantly changing the total contact force. For example, the system can instruct the robot to apply a small sinusoidal force fluctuation with a peak-to-peak value of ±0.2 N and a frequency of 0.1 Hz along the direction perpendicular to the contact surface while maintaining an average contact force of 2 N, or attempt a small displacement (e.g., ±0.5 mm) along a direction within the contact plane under the constant contact force. While executing these micro-perturbations, the force / torque sensor accurately records the dynamic response of the object to these small force / displacement inputs (i.e., the relationship between force and displacement or velocity) at an extremely high frequency (e.g., 2000 Hz). These additional high-frequency sensor data reflecting the local mechanical properties of the object, together with all the sensor data and robot state records collected during the entire constant force maintenance stage, after integration and timestamp calibration, jointly constitute the static contact characteristic data. These data reflect the basic physical properties of the object such as surface hardness, local stiffness, and damping under static or quasi-static loading conditions.

[0057] Preferably, the dynamic interaction and multi-modal signal acquisition in step S2 include:

[0058] Based on the static contact characteristic data, perform grasping force increasing control to obtain the characteristic curve of the grasping process;

[0059] According to the grasping process characteristic curve, perform lifting gravity response analysis to obtain the lifting gravity response curve;

[0060] According to the lifting gravity response curve, evaluate the suspended stability of the item to obtain the suspended stability data;

[0061] Based on the suspended stability data, perform perception action execution and enhance micro-perturbations to obtain perturbation action response data;

[0062] Collect sensor array data according to the perturbation action response data;

[0063] Perform multi-modal data time alignment on the sensor array data to obtain a time-synchronized data stream;

[0064] Integrate the perturbation action response data and the time-synchronized data stream to obtain dynamic interaction response data;

[0065] Perform signal synchronization and noise reduction on the dynamic interaction response data to obtain a noise-reduced signal data packet;

[0066] Perform data encapsulation and transmission on the noise-reduced signal data packet to obtain interactive perception data.

[0067] In the embodiment of the present invention, according to the preset target grasping force in the grasping execution plan (for example, set to 15 N for a cardboard box), the robot control system starts the grasping force increasing control. The control mode switches from the previous constant force holding to a controlled force / position hybrid control or pure force control. The system instructs the end effector (gripper) of the robot to gradually increase the clamping force or supporting force on the item. The force increasing process can adopt a linear increasing mode (for example, linearly increase the force from 2 N to 15 N within 0.5 seconds), or a segmented increasing mode, or even an adaptive force increasing strategy based on the mechanical response of the item (preliminarily analyzed from the static contact characteristic data) to avoid excessive force. During this entire force increasing process, the high-precision force / torque sensor on the robot board continuously collects the three-axis force (Fx, Fy, Fz) and three-axis torque (Tx, Ty, Tz) data between the gripper and the item at a high frequency (for example, 1000 Hz). These force / torque measurement values that change with time accurately reflect the deformation, load-bearing capacity, and initial response of the internal structure of the item packaging under the action of external forces, and are recorded to form the grasping process characteristic curve.

[0068] After confirming that the grasping is stable, the robot control system performs an action to lift the object vertically upward. The lifting motion adopts a controlled speed curve (for example, first accelerating to 0.2 m / s at an acceleration of 0.1 m / s² and then maintaining a constant speed). Throughout the process when the object completely leaves the support surface (such as a shelf) and enters the suspended state, the force / torque sensor continues to collect data at a high frequency. The system analyzes the force / torque data in this stage, paying particular attention to the force Fz in the direction against gravity (usually the negative direction of the Z-axis in the robot base coordinate system). When the object accelerates upward, Fz will be greater than the actual weight of the object; when the object rises at a constant speed and is stably suspended, the average value of Fz will approach the total gravity of the object (i.e., mass M × gravitational acceleration g, Fz≈Mg). The changes in force / torque during the lifting process, especially the stable value of the vertical force and the fluctuations of the horizontal force / torque, reflect the total mass of the object, the position of the center of mass, and whether the grasping is balanced. These force / torque time series data reflecting the object's gravity and grasping balance constitute the lifting gravity response curve.

[0069] After the object is completely suspended and stops vertical movement, the robot maintains the grasping posture to keep the object stationary and suspended. The system analyzes the force / torque sensor data in this stationary suspension stage. The evaluation indicators include: the fluctuation amplitude of the vertical force Fz (for example, calculating the standard deviation of Fz ), which reflects whether the grasping shakes or there are continuous small movements inside the object; the average values and fluctuation amplitudes of the horizontal force components Fx, Fy and the torque components Tx, Ty, Tz (especially Tz around the vertical axis), which reflect whether the grasping point deviates from the center of mass of the object and whether the object rotates or tilts during grasping. If the average values of Fx, Fy or Tx, Ty are significantly non-zero, or their fluctuation amplitudes ( , , , ), are large, it indicates that the grasping is unstable or the center of mass of the object deviates from the grasping center. If the fluctuation of Tz ( ) is large, it indicates that there are rotating components or liquid sloshing inside the object. The system calculates these statistics and compares them with the preset stability thresholds. These quantified stability evaluation results (for example, = 0.05N, average Fx = 0.1N, = 0.02 Nm), constitute the suspended stability data.

[0070] Based on the safety constraint action plan generated in step S14, the system instructs the robot control system to execute the perception actions specific to the item (e.g., slight shaking or squeezing). During the execution of the perception actions, the robot joint control system or the end effector force control system drives the robot to act precisely according to the parameters adjusted for safety in the plan (e.g., shaking amplitude of 5°, frequency of 2 Hz; squeezing pressure of 10 N). To enhance the sensor signals, the system can superimpose and execute micro-perturbations on the basis of the main perception actions. For example, while squeezing, a tiny, high-frequency (e.g., 100 Hz) force pulse sequence is applied to the squeezing surface, or while shaking, a tiny vibration along different axes is superimposed. During the entire set duration (e.g., 1.5 seconds) of executing the perception actions and superimposing the micro-perturbations, all the activated multi-modal sensors (force / torque sensors, vibration sensors, microphones) continuously collect the raw data at their configured high frequencies (e.g., force sensor at 1000 Hz, vibration sensor at 500 Hz, microphone at 16 kHz). These sensor data streams, which record the physical responses of the item to the dynamic interactions and micro-perturbations actively imposed by the robot, constitute the perturbation action response data.

[0071] When performing the perception actions and enhancing the micro-perturbations based on the suspended stability data, the on-board sensor array of the robot, including at least force / torque sensors, vibration sensors, and microphones, synchronously and at a high frequency collects the response signals of the item to these dynamic inputs. These sensors are distributed at different positions on the robot's grasping arm or the end effector, forming a sensor array. Each sensor works independently, collects the data stream of its specific modality, and generates data packets with local timestamps. The set of raw data streams collected in parallel from the sensor array is the sensor array data.

[0072] Due to different sensors using independent internal clocks or having different data transmission delays, their original timestamps are not exactly the same. For subsequent multi-modal data fusion analysis, it is necessary to precisely align these data streams to the same global time axis. The system uses a high-precision time synchronization mechanism. One method is that the data acquisition of all sensors is triggered by the synchronization pulses sent by the robot main control system to ensure the simultaneity of sampling. Another method is to utilize the hardware timestamps included in each sensor data packet and perform post-processing alignment in the main control system or the data processing unit. Post-processing alignment can adopt an interpolation algorithm. According to the local timestamp of each data point, it is mapped to the unified global time axis and resampled to generate a multi-modal data vector aligned at each global time point. The alignment accuracy generally needs to reach the millisecond level or even the sub-millisecond level to retain the transient characteristics of the signal. The processed data, in which the data points from different sensors precisely correspond on the time axis, forms the time-synchronized data stream.

[0073] The system integrates other relevant data generated during the dynamic phase (grasping, lifting, hovering, sensing actions) of this inventory operation, including the grasping process characteristic curve, the lifting gravity response curve, the hovering stability data, as well as the precise motion instruction parameters and the actual execution trajectory record when the robot performs sensing actions. The system organizes all this data, including the original and preliminarily processed (such as time alignment) sensor data, the robot state data, and the preliminary features or evaluation results calculated based on these data, into a unified and structured data packet. This data packet contains a complete record of all the dynamic interactions between the robot and the item and the multi-modal sensor responses generated during the period from when the item is grasped to when the sensing actions are completed. This structured data packet that aggregates all the information related to the dynamic phase is the dynamic interaction response data.

[0074] Receive the dynamic interaction response data. Although time alignment was performed in the previous step, this step can perform more refined signal synchronization verification to ensure that all modal data remains aligned on a micro time scale. More importantly, this step performs signal denoising processing. For force / torque and vibration signals, digital filters (e.g., Butterworth filters) are applied to remove high-frequency random noise and specific frequency interferences introduced by the robot's own motion or environmental vibrations (e.g., using notch filters to remove the known resonance frequencies of the robot's motors or joints). For acoustic signals, techniques such as spectral subtraction or non-negative matrix factorization are applied, using the environmental reference noise data collected in step S2.2 (approach and positioning preparation) to separate and suppress the background noise from the acoustic signals generated by item interactions. The denoising algorithm parameters are optimized according to the sensor type and environmental conditions. The denoising process aims to improve the signal-to-noise ratio and highlight the inherent response characteristics of the item itself. The multi-modal sensor data stream after refined synchronization and denoising processing is encapsulated into a denoised signal data packet.

[0075] Perform the final encapsulation of the denoised signal data packet. The encapsulation structure includes: the unique item identifier (Item_ID), the robot identifier (Robot_ID) that performs the inventory operation, the start timestamp of this interaction sensing process, the type and parameters of the sensing actions performed, and a standardized format block containing the denoised force / torque, vibration, acoustic, etc. sensor time series data. The sensor data can be compressed using compression algorithms (e.g., lossless compression algorithms such as LZ4 or lossy compression algorithms such as Opus for audio, provided that key features are retained) to reduce the data volume. The encapsulated data packet is named interaction sensing data and is transmitted via the warehouse internal network (e.g., a secure communication channel based on the TCP / IP protocol) to the central processing system or a dedicated data analysis server for subsequent intelligent material sensing analysis (step S3). An error detection and retransmission mechanism is adopted during the transmission process to ensure data integrity.

[0076] Preferably, the separation of the sensing signals in step S3 includes:

[0077] Separating and preprocessing the interactive sensing data to obtain a classified signal feature set;

[0078] Extracting mechanical characteristic parameters from the classified signal feature set to obtain a mechanical parameter feature table;

[0079] Generating an acoustic feature spectrogram from the classified signal feature set to obtain an acoustic feature spectrogram.

[0080] In the embodiment of the present invention, according to the predefined format and identifier in the data packet, the data streams of different sensor modalities are separated and extracted to form three independent original signal sets: a force / torque signal set, a vibration signal set, and an acoustic signal set. Then, modal-specific preprocessing is performed on each signal set. For the force / torque signal set, a fourth-order Butterworth low-pass filter is applied, and the cut-off frequency is set to 50 Hz to filter out the high-frequency noise brought by high-speed sampling. For the vibration signal set, a fourth-order Butterworth band-pass filter is applied, and the passband range is set to 20 Hz to 200 Hz to focus on the characteristic frequency band generated by the vibration of the item structure and the movement of the contents. For the acoustic signal set, the short-time Fourier transform (STFT) method is used, and the Hanning window function is used. The window length is set to 1024 sampling points, and the overlap rate is 50% to generate the time-frequency representation of the acoustic signal, that is, the spectrogram. At the same time, amplitude normalization processing is performed on all the filtered and transformed signal data. For example, the Z-score normalization method is used to convert the signal values into a distribution with a mean of 0 and a standard deviation of 1, so that the data between different modalities and different items is comparable in the subsequent process. All these data sets that have been separated and subjected to modal-specific preprocessing are organized and named as the classified signal feature set, which contains a force / torque signal array, a vibration signal array, and an acoustic spectrogram matrix.

[0081] Extract parameters reflecting the mechanical properties of an item from the force / torque signal array. For example, the average value of the vertical force component (e.g., Fz) during the stage of uniform lifting of the item is used to estimate the equivalent mass of the item (mass ≈ average Fz / gravitational acceleration g). Analyze the curve of force versus gripper displacement during the increasing process of the grasping force, and calculate its slope, which reflects the local stiffness of the item's packaging. Analyze the slope of the curve of the vertical force (Fz) versus the robot's angular acceleration (α) during the execution of the shaking action, which reflects the equivalent moment of inertia of the movable medium (such as liquid, particles) inside the item and can be used as an indicator of the internal filling amount. Analyze the curve of the squeezing force versus the squeezing displacement during the execution of the squeezing action, and calculate its non-linear characteristics or elastic recovery ratio, which reflects the elasticity and compressive resistance of the packaging. From the vibration signal array, after the item is tapped or shaken, analyze the decay rate of the vibration amplitude, and calculate the energy decay coefficient of the vibration signal, which reflects the damping characteristics of the internal structure of the item. All these calculated quantitative values, such as the estimated mass (unit: kg), packaging stiffness (unit: N / mm), equivalent moment of inertia (unit: kg· ), elastic recovery ratio (dimensionless quantity), energy decay coefficient (unit: 1 / s), are organized into a structured list or table, with each row corresponding to a set of mechanical property parameters of an item, forming a mechanical parameter feature table.

[0082] Extract acoustic features reflecting the internal state of the item from the acoustic spectrogram matrix. For example, calculate the total energy of a specific low-frequency band (such as 5 Hz - 15 Hz), which is related to the intensity of liquid sloshing. Calculate the total energy of another specific frequency band (such as 50 Hz - 100 Hz), which is related to particle collisions or sliding. Analyze the change of the spectral centroid (representing the "brightness" or the center of frequency distribution of the spectrum) of the spectrogram over time, which reflects the changing characteristics of the sound. Calculate the spectral flux (measuring the change in the spectral shape between consecutive frames) of the spectrogram, which reflects the dynamics of the sound. From the spectrum of the vibration signal, detect whether there are significant resonance peaks and identify their main frequencies and bandwidths, which reflect the natural vibration modes of the whole item or its internal structure. Analyze the amplitude envelope of the acoustic signal and calculate the onset time, duration, and decay time of the sound. For example, the sound of liquid sloshing usually lasts for a long time and decays slowly, while the sound of solid impact decays rapidly. The system integrates these extracted acoustic and vibration features, such as the energy ratio of a specific frequency band (dimensionless), spectral centroid (unit: Hz), spectral flux (dimensionless), resonance frequency (unit: Hz), resonance peak bandwidth (unit: Hz), sound duration (unit: s), etc., into a multi-dimensional feature vector or matrix. This vector or matrix describes the acoustic and vibration response characteristics of the item in a quantitative way and can be regarded as the acoustic feature map of the item for subsequent item identification and internal state judgment.

[0083] Preferably, the generation of the acoustic feature map includes:

[0084] Performing acoustic and vibration time-frequency conversion on the classified signal feature set to obtain a time-frequency energy matrix;

[0085] Dividing the material feature frequency bands based on the time-frequency energy matrix to obtain a frequency band statistical feature table;

[0086] Performing spectral centroid and morphology calculations on the time-frequency energy matrix to obtain a spectral morphology parameter sequence;

[0087] Performing time-varying analysis of the sound energy envelope on the classified signal feature set to obtain an envelope dynamic parameter table;

[0088] Performing acoustic fingerprint matching calculations on the frequency band statistical feature table, the spectral morphology parameter sequence, and the envelope dynamic parameter table to obtain a material acoustic feature matching table;

[0089] Extracting filling rate acoustic markers based on the material acoustic feature matching table and the envelope dynamic parameter table to obtain a filling state acoustic index table;

[0090] Performing abnormal resonance and acoustic deviation detection on the time-frequency energy matrix to obtain an abnormal acoustic feature table;

[0091] Generating an acoustic feature map according to the abnormal acoustic feature table and the filling state acoustic index table.

[0092] In the embodiments of the present invention, time-frequency analysis, spectral feature calculation, time-varying envelope analysis, and pattern matching techniques are comprehensively used to extract and quantify the acoustic features reflecting the internal structure and state of the article from these signals. First, time-frequency conversion is performed on the sound and vibration signals to generate a matrix describing the distribution of signal energy in time and frequency. Then, based on this matrix, the system identifies and statistically analyzes the energy distribution of the frequency response frequency bands unique to different material types. At the same time, the statistical moments (such as centroid, bandwidth) and morphological parameters of the spectrum in each time frame are calculated to describe the shape characteristics of the spectrum. In addition, the envelope line of the total energy of the sound signal changing with time is analyzed to extract its dynamic parameters. These calculated quantitative features are compared with a preset acoustic fingerprint template database to identify the acoustic type of the article. On this basis, the acoustic indicators related to the filling rate are further analyzed, and whether there are abnormal resonances or acoustic deviations inconsistent with the expectations is detected. Finally, these multi-dimensional and quantitative acoustic analysis results are integrated to generate an acoustic feature map of the article.

[0093] For acoustic signals, the system uses the Short-Time Fourier Transform (STFT) method for time-frequency analysis. The analysis window type is set to a Hamming window, with a window length of 1024 sampling points and a frame shift (the interval between the starting points of adjacent windows) of 512 sampling points (i.e., 50% overlap). The Fast Fourier Transform (FFT) is performed on the signal segment within each time window, its spectrum is calculated, and the square of the magnitude is taken to obtain the energy spectrum. The energy spectra of consecutive time windows are arranged in chronological order to form a two-dimensional matrix E_acoustic, where the rows represent time frames and the columns represent frequency bins, and the matrix element E_acoustic[t,f] represents the energy at frequency f corresponding to time frame t. For vibration signals, the STFT method is also used, with a window length set to 256 sampling points and a frame shift of 128 sampling points. The FFT is performed on the vibration signal of each time window (e.g., the vector norm of triaxial acceleration) and the energy spectrum is calculated to obtain the time-frequency energy matrix E_vibration of the vibration signal. The system can either merge E_acoustic and E_vibration into an overall time-frequency energy matrix containing the energy distributions of different modalities or process them as two independent matrices, but both are marked with the same global timestamp.

[0094] The system internally stores a library of defined characteristic frequency bands for different material types. For example, the frequency band library defines: the liquid sloshing frequency band L (5Hz - 15Hz), the particle sliding / collision frequency band G (50Hz - 100Hz), the hard structure resonance frequency band (150Hz - 250Hz), the packaging material resonance frequency band (80Hz - 120Hz). The system traverses each time frame of the time-frequency energy matrix. For each time frame, the system calculates the total energy or average energy within each predefined characteristic frequency band. For example, for time frame t and the liquid sloshing frequency band L, calculate ∑E[t,f], where f belongs to the frequency range corresponding to the frequency band L. In addition to the total energy, statistical quantities such as the energy peak and the standard deviation of the energy within each frequency band can also be calculated. These statistical quantities reflect the activity level of the item within different characteristic frequency ranges at a specific time point. The system organizes these statistical quantities calculated by time frame and frequency band into a table or matrix, such as a three-dimensional matrix [time frame × frequency band × statistic type], to form the frequency band statistical feature table.

[0095] Traverse each time frame of the time-frequency energy matrix. For the spectrum corresponding to each time frame t (e.g., the acoustic energy spectrum E_acoustic[t,:]), the system calculates the statistical quantities describing its shape. Calculate the spectral centroid, with the formula =(∑ ×E[t, ) / (∑E[t, )), where is the frequency corresponding to the frequency bin. Calculate the Spectral Spread, which reflects the degree of dispersion of energy on the frequency axis. Calculate the Spectral Skewness and Spectral Kurtosis, which reflect the symmetry and kurtosis of the energy distribution. Calculate the Spectral Flatness, which reflects whether the spectrum is biased towards noise (flat) or towards tone (sharp). These calculations yield a set of values for each time frame, forming a parameter vector that varies with time. Arrange the parameter vectors of all time frames in sequence to form a matrix [time frame × morphological parameter type], which is the spectral morphological parameter sequence. For example, the sequence contains , , and other trajectories of parameters varying with time t.

[0096] Calculate the energy envelope of the acoustic signal. One method is to rectify the signal (take the absolute value) and then smooth the signal through a low-pass filter (e.g., cut-off frequency 10 Hz) to obtain a curve reflecting the variation of sound loudness with time. The system analyzes the dynamic characteristics of this envelope. The extracted parameters include: total duration (the time interval from the start to the end of the sound, e.g., the time point when the energy envelope drops below 10% of the peak); attack time (the time from the start point of the sound to the envelope reaching the peak); decay time (the time required for the envelope to drop from the peak to a certain ratio (e.g., 50% or 10% of the peak)); peak amplitude of the envelope; RMS (root mean square) value of the envelope; characteristics of the envelope shape (e.g., whether it contains multiple distinct peaks and the intervals between the peaks). These calculated quantitative values, such as total duration (unit: s), attack time (unit: s), decay time (unit: s), peak amplitude (unitless or relative unit), are organized into a table to form the envelope dynamic parameter table.

[0097] Integrate these features from different analysis dimensions to form a high-dimensional acoustic feature vector of the item. For example, the results averaged by time frames in the frequency band statistical feature table, the average value of the spectral morphology parameter sequence or the snapshot at a specific time point, and all the parameters in the envelope dynamic parameter table can be concatenated to form a comprehensive feature vector F_acoustic. Internally, the system stores a database of acoustic fingerprints of known item / material types. This database contains typical acoustic feature vectors or feature distribution models (such as Gaussian mixture models) corresponding to various known item types. The system uses a pattern matching algorithm to compare the comprehensive feature vector F_acoustic of the current item with each known acoustic fingerprint in the database. Multiple methods can be used for the matching calculation, such as: calculating the Euclidean distance or cosine similarity between F_acoustic and each typical feature vector in the database; if the database stores feature distribution models, calculating the likelihood of F_acoustic under each model. The system sorts according to the matching degree or likelihood to obtain a list of known item types most similar to the acoustic features of the current item and their corresponding matching scores (such as a similarity score of 0 - 1). The type with the highest score is considered the preliminary recognition result. The system records this recognition result, including information such as the identified item type identifier, the highest matching score, and the second-highest matching score, to form a material acoustic feature matching table.

[0098] Check the item type identified in the material acoustic feature matching table to determine whether this type belongs to container items (such as liquid containers, particle containers) for which the filling rate needs to be estimated. If not, skip this step. If it is a container item, the system extracts acoustic metrics highly correlated with the filling state from the envelope dynamic parameter table and / or the frequency band statistical feature table according to the specific item type matched. For example, for a liquid container, extract the energy peak value, energy decay time, and the duration of the envelope line in the sloshing frequency band (5Hz - 15Hz). For a particle container, extract the energy in the particle collision frequency band (50Hz - 100Hz), the damping coefficient of the vibration signal, and the rapid decay characteristic of the sound envelope line. Internally, the system stores mapping relationship models between acoustic metrics and filling rates for different types of containers and contents. These models can be regression models trained based on experimental data (such as a linear regression model calculating Fill_Rate = a×Peak_Energy + b×Decay_Time + c) or pre-calibrated lookup tables. The system inputs the extracted relevant acoustic metrics into the corresponding models to calculate quantitative metrics reflecting the filling state. These metrics are directly the estimated filling rate percentages or some intermediate quantities strongly correlated with the filling rate (such as equivalent sloshing mass, equivalent particle density). The system records these calculated acoustic metrics related to the filling state and / or the preliminary filling rate estimation values to form a filling state acoustic metric table.

[0099] Obtain the item types identified in the material acoustic feature matching table and query the "normal" acoustic feature templates or statistical distributions for items of this type (e.g., the energy range in key frequency bands and the expected resonance frequencies under normal conditions). The system compares the time-frequency energy matrix of the current item with the normal template to detect any anomalies. The detection methods include: ① Abnormal resonance detection: Search the entire frequency spectrum for significant energy peaks. Whether the frequencies corresponding to these peaks do not match the natural frequencies of normal items, or whether their Q factors (the sharpness of the peak, reflecting damping) are abnormally high or low. For example, if a solid item shows a sharp resonance peak in a certain high-frequency band, it indicates the presence of an internal cavity or loose components. ② Acoustic deviation detection: Calculate the difference metric between the time-frequency energy distribution of the current item and the normal template (e.g., using KL divergence or spectral distance). If the difference exceeds a preset threshold, it is marked as having an acoustic deviation. Detect whether there are sudden and short-lived high-energy events, which are abnormal noises generated by internal component fractures or foreign object collisions. The system quantifies these abnormal manifestations. For example, record the frequencies, amplitudes, and bandwidths of abnormal resonances, record the time points and intensities of abnormal transient noises, and calculate the overall acoustic deviation score. Organize these quantified abnormal information into a table or list to form an abnormal acoustic feature table.

[0100] Integrate all key analysis results regarding the acoustic characteristics of the item. These results include: the item types identified in the material acoustic feature matching table (and their confidence levels), the filling levels or related indicators reflected in the filling state acoustic index table, and any anomalies detected in the abnormal acoustic feature table (such as abnormal resonances, abnormal noises, overall deviations). The system organizes this information into a structured data object, which is the acoustic feature map of the item. The acoustic feature map is a multi-dimensional representation that not only contains the item type recognition results but also includes in-depth analysis conclusions about its "internal sound" and a quantitative assessment of the filling state. By comprehensively processing the data in the abnormal acoustic feature table and the filling state acoustic index table, a more complete acoustic feature description is generated, ensuring that the map contains both the abnormal state information of the item and accurate filling level estimation results. For example, the acoustic feature map can contain fields: item ID, identified acoustic type (e.g., "liquid container"), type confidence level (e.g., 0.92), filling state indicators (e.g., "estimated filling rate: 75%", "related indicators: high sloshing energy, slow decay"), internal state anomaly marker (e.g., "anomaly present"), anomaly details (e.g., "detected high-frequency abnormal resonance @ 180 Hz, abnormally high Q factor, suspected internal structure anomaly"). This map integrating the abnormal detection results and the filling state assessment provides a comprehensive and detailed acoustic perception-based basis for subsequent inventory information updates and status assessments.

[0101] Preferably, the quantity / filling amount estimation and perception report generation in step S3 include:

[0102] Extract the filling-related parameter set from the items identified as container types in the item type determination result;

[0103] Match the filling-related parameter set with the pre-established physical model parameter library to obtain the estimation model selection result;

[0104] Calculate the liquid filling amount based on the estimation model selection result to obtain the calculated value of the liquid filling rate;

[0105] Estimate the number of particles based on the estimation model selection result to obtain the calculated value of the particle filling rate;

[0106] Estimate the count of solid items based on the estimation model selection result to obtain the estimated value of the item quantity;

[0107] Evaluate the estimation reliability of the calculated value of the liquid filling rate, the calculated value of the particle filling rate, and the estimated value of the item quantity to obtain the estimation reliability score;

[0108] Calibrate the estimation reliability score with historical data and generate the filling amount estimation result;

[0109] Generate a comprehensive report on the item type determination result, the internal state assessment form, and the filling amount estimation result to obtain the material perception feature report.

[0110] In the embodiments of the present invention, the analysis module receives the item type determination result, the internal state evaluation form, and various characteristic data (such as mechanical parameter characteristic table, acoustic characteristic map, filling state acoustic index table, etc.) generated in the previous steps (S31 - S33). The system first identifies from the item type determination result the items for which quantity or filling amount estimation is required, that is, those items determined to be of container type (such as liquid containers, particle containers) or types containing countable solid items. For these items, the system extracts a series of quantization parameters related to the mass, volume, density, sloshing characteristics, vibration damping, acoustic response, etc. of the item's contents from the mechanical parameter characteristic table and the filling state acoustic index table, forming a filling - related parameter set. Then, according to the specific type of the item and the characteristics of the filling - related parameter set, the system searches and matches the most suitable model or algorithm for estimation in the pre - established physical model parameter library, obtaining the estimation model selection result. Based on the selected model, the system performs corresponding estimation calculations on liquid, particle, or solid items respectively, obtaining preliminary calculated values (such as liquid filling rate calculated value, particle filling rate calculated value, item quantity estimated value). To evaluate the reliability of these estimation results, the system calculates an estimation reliability score considering factors such as sensor data quality, item type recognition confidence, and information consistency provided by different sensing modalities (mechanical, acoustic). The system calibrates the estimation reliability score and / or the preliminary estimation value using historical inventory data and the corresponding sensing estimation results, generating a more accurate filling amount estimation result, which includes the final estimated value and the estimated error range. Finally, the system integrates the item type determination result, the internal state evaluation form, and the filling amount estimation result into a structured document, generating a complete material perception feature report.

[0111] Check whether the identified item type belongs to the preset list of "estimable filling" or "estimable quantity" categories. If the item type is in the list, the system extracts key parameters related to the content state and quantity from the mechanical parameter characteristic table (including estimated mass M_est, estimated moment of inertia I_est, packaging stiffness K_package, vibration damping coefficient ζ, etc.) and the filling state acoustic index table (including sloshing frequency band energy E_shake, particle frequency band energy E_granular, sound energy decay time τ_decay, specific resonance frequency f_resonance, etc.) corresponding to the item. For example, for "liquid container_model A", extract M_est, I_est, E_shake, τ_decay; for "particle bag_model B", extract M_est, ζ, E_granular; for "small box_contains 10 solid pieces", extract M_est, f_resonance (reflecting internal looseness or counting). The set of these extracted quantization values is organized into a filling - related parameter set for each item to be estimated.

[0112] Based on the identification type of the item, search for a matching model in the physical model parameter library. For example, if the item is identified as "liquid container_model A", the system searches for a list of models applicable to "liquid container_model A" (e.g., , , ). The system further checks which input parameters required by the models are included in the filling-related parameter set and evaluates the parameter quality (e.g., the signal-to-noise ratio of the sensor signal). The system selects the model with the highest priority, complete required parameters, and qualified quality as the estimation model for the current item. If multiple models are matched, the system can select the optimal one according to preset rules (e.g., select the model based on more perceptual modality data), or mark it as possible to use multiple models for redundant estimation. The selected model identifier and its required parameter list constitute the estimation model selection result.

[0113] Extract all input parameters required by the model from the filling-related parameter set (e.g., M_est, I_est, E_shake, τ_decay). The system queries the item master data to obtain the known attributes of the liquid container, such as the empty container mass M_empty, the total volume V_total, the container height H_container, and the liquid density ρ_liquid. The system applies the liquid filling amount calculation formula or algorithm specified in the estimation model selection result. For example, if the model is selected, its formula is: estimated liquid mass M_liquid_est = (M_est, M_empty) and estimated liquid moment of inertia I_liquid_est = (I_est, I_empty), where , is the conversion function, and I_empty is the moment of inertia of the empty container. Then, according to the liquid sloshing physical model, estimate the filling height h_est = (I_liquid_est, M_liquid_est, H_container). Finally, calculate the liquid filling rate: Fill_Rate_Liquid = (h_est / H_container) × 100%. If the model uses acoustic parameters, the formula is: Fill_Rate_Liquid = (E_shake, τ_decay, V_total). The system performs the calculation to obtain a percentage value, which is the calculated value of the liquid filling rate.

[0114] Extract all the input parameters required by the model from the filling-related parameter set (e.g., M_est, ζ, E_granular). The system queries the item master data to obtain the known attributes of the granular container, such as the empty container mass M_empty, the total volume V_total, and the bulk density range ρ_bulk_range of the granular material. The system applies the calculation formula or algorithm for the number of granules / filling amount specified in the estimation model selection result. For example, if the model is selected, its formula is: Estimated granular mass M_granular_est = M_est - M_empty. Estimated granular volume V_granular_est = M_granular_est / ρ_bulk_average, where ρ_bulk_average is the typical bulk density of the granular material. Finally, calculate the granular filling rate: Fill_Rate_Granular = (V_granular_est / V_total) × 100%. If the model uses acoustic and vibration parameters, the formula is: Fill_Rate_Granular = (ζ, E_granular, V_total). The system performs the calculation to obtain a percentage value (filling rate) or a numerical value (if it is a discrete granule count), that is, the calculated value of the granular filling rate.

[0115] Extract all the input parameters required by the model from the filling-related parameter set (e.g., M_est, f_resonance, internal structure characteristic flag). The system queries the item master data to obtain the known attributes of the solid item, such as the mass M_single of a single standard item. The system applies the calculation formula or algorithm for the solid item count specified in the estimation model selection result. For example, if the model is selected based on mass estimation, the formula is: Estimated number of items N_est = round((M_est - M_empty_package) / M_single), where M_empty_package is the empty mass of the package. If the model is selected based on internal resonance or impact counting (when shaking causes the internal items to collide with each other to generate detectable impacts), the formula is: N_est = (f_resonance, Number_of_Impacts), where Number_of_Impacts is the number of impact events detected from the acoustic / vibration data. The system performs the calculation to obtain a numerical value, that is, the estimated value of the number of items.

[0116] Calculate the reliability score for each estimation result. The basis for scoring includes: ① The signal-to-noise ratio of the input sensor data: The higher the signal-to-noise ratio, the higher the reliability score. ② The confidence level of item type recognition: The higher the confidence level of type determination, the higher the reliability score for the estimation using the corresponding model of that type. ③ The consistency of multi-modal data: For example, if the indication directions of the estimation quality and the acoustic analysis results (such as sloshing energy) for the filling rate are the same, the reliability score increases; if they are inconsistent, the reliability score decreases. ④ The inherent accuracy of the model used: The historical verification accuracy of each model is recorded in the physical model parameter library; the higher the accuracy of the model used, the higher the reliability score. ⑤ The significance of the original signal features: For example, if the characteristic signal generated by liquid sloshing is very weak, even if it is identified as liquid, the reliability score for its filling rate estimation is relatively low. The system synthesizes these factors and calculates a value between 0 and 1 through a preset scoring function Reliability_Score= (Signal_SNR,Type_Confidence,Modality_Consistency,Model_Precision,Feature_Salience), where 1 represents the highest reliability. The system generates corresponding estimation reliability scores for each estimation performed (liquid filling rate, particle filling rate, solid quantity).

[0117] Access the historical inventory database, which stores the data of past manual inventories or high-precision measurements, as well as the estimation results and various characteristic data obtained by sensing methods at that time. The system uses these historical data to establish a calibration model. The calibration model can be a simple linear regression model (for example, the final estimated value = a × preliminary estimated value + b), or a more complex machine learning model, which takes the preliminary estimated value, the estimation reliability score, the item type, and some original characteristic data as inputs and outputs the corrected final estimated value and the estimated error range. The goal of model training is to minimize the difference between the corrected estimated value and the historical true value. The system corrects the preliminary estimated value using the corresponding historical data calibration model according to the type of the current item and the estimation reliability score. For example, if the historical data shows that for a certain liquid type, when the reliability score is 0.7, the estimated value is usually 5% lower, then the system increases the preliminarily calculated filling rate by 5%. The calibration model also outputs an estimated error range, such as ±5%. The final filling quantity estimation result includes the calibrated estimated value (for example, "Liquid filling rate: 78%" or "Number of items: 9"), the estimated error range (for example, "±5%" or "±1"), and the estimation reliability score.

[0118] Integrate these analysis results into a structured material perception feature report document. The report structure includes, but is not limited to: report generation time, item unique identifier (Item_ID), item location in the warehouse (Location_ID), result of item type determination based on perception (including the identified type and confidence level), result of internal state assessment based on perception (including status markers such as "normal", "abnormal", specific abnormal types such as "abnormal liquid sloshing", "suspected internal looseness", and abnormal severity score), result of quantity / filling level estimation based on perception (including the estimated value, estimated error range, and estimation reliability score). The report can also include meta-information of the original perception data (such as collection time, type of perception action performed) and recommended actions generated by the system based on these results (for example, "inventory is normal", "manual verification of quantity is required", "move to the quality inspection area"). The system outputs this structured document containing comprehensive perception analysis results, which is the material perception feature report.

[0119] Preferably, the inventory information synchronization and update in step S4 includes:

[0120] Receive and verify the material perception feature report to obtain a verification result record;

[0121] Compare the material perception feature report with the inventory data according to the verification result record to obtain a data difference analysis table;

[0122] Update the inventory records according to the data difference analysis table to obtain an update data packet;

[0123] Evaluate the update permissions and priorities of the update data packet to obtain a hierarchical update plan;

[0124] Execute database transactions according to the hierarchical update plan to obtain a database update result.

[0125] In the embodiments of the present invention, the format, integrity, and content of the report are verified to ensure its validity, and a verification result record is obtained. Based on the verified report, the system compares it with the records in the current inventory database to identify whether there are differences in information such as item type, quantity / filling level, and internal state, and generates a data difference analysis table. The system prepares instructions to modify the inventory database according to these differences and preset update rules to form an update data packet. The system then evaluates the permission requirements and business urgency of each modification in the update data packet to determine which updates can be executed automatically, which require manual approval, and assigns priorities to generate a hierarchical update plan. Finally, the system executes the approved update operations according to the hierarchical update plan through the database transaction processing mechanism to ensure data consistency, obtains a database update result, and records the update details.

[0126] The material perception feature report is transmitted in a structured format (e.g., JSON object), including fields such as item identifier (Item_ID), location identifier (Location_ID), sensing timestamp (Sensing_Timestamp), sensed item type (Sensed_Item_Type), internal state assessment result (Sensed_Internal_State, e.g., including status code and description), estimated quantity / fill level value (Sensed_Quantity_Value) and its unit, estimated error range (Sensed_Quantity_Error), estimated reliability score (Sensed_Reliability_Score), etc. The data integration service module executes a series of validation rules on the received report: ① Format validation: Check whether the data structure conforms to the predefined JSON Schema; ② Required field check: Confirm whether all mandatory fields (such as Item_ID, Location_ID, Sensing_Timestamp) exist and are not empty; ③ Identifier validity check: Query the item master data table to confirm whether the Item_ID is valid, and query the location table to confirm whether the Location_ID is valid; ④ Timestamp validity check: Verify whether the Sensing_Timestamp is within a reasonable window of the current time (e.g., not exceeding the past 5 minutes); ⑤ Data range check: Verify whether the numerical fields (such as Sensed_Quantity_Value, Sensed_Reliability_Score) are within their valid ranges (e.g., the reliability score is between 0 and 1, and the fill rate is between 0% and 100%). Any validation failure will be recorded, and a validation result record containing the reason for failure will be generated. The reports that pass the validation are marked as ready for further processing, and a validation result record containing a success flag is generated.

[0127] If the verification result record indicates that the report verification is successful, the system extracts the Item_ID and Location_ID from the material perception feature report. The system obtains the current inventory record of the item at the specified location from the Inventory_Main table of the warehousing inventory management system by executing a database query statement (e.g., SELECT Current_Qty, State, Item_Type, Last_Inventory_Time FROM Inventory_Main WHERE Item_ID=? AND Location_ID=?). The system compares the sensed data in the report with the existing data in the database item by item: ① Item type comparison: Compare whether Sensed_Item_Type is consistent with Inventory_Main.Item_Type; ② Quantity / fill level comparison: Calculate the absolute difference |Sensed_Quantity_Value - Inventory_Main.Current_Qty| and the relative difference percentage (|Sensed_Quantity_Value - Inventory_Main.Current_Qty| / Inventory_Main.Current_Qty) × 100% (if Current_Qty > 0) between Sensed_Quantity_Value and Inventory_Main.Current_Qty; ③ Internal state comparison: Compare the status code or description of Sensed_Internal_State with the Inventory_Main.State field; Specifically mark the cases where the status changes from "normal" to "abnormal"; ④ Inventory time record: Record the Sensing_Timestamp in the report as the new candidate value for the last inventory time. All these comparison results, including the difference values, difference percentages, status change marks, etc., are structurally recorded to form a data difference analysis table for the item.

[0128] Based on a preset update rule set that is based on the type and magnitude of differences, determine which specific database update instructions to generate. For example, the rule set includes: Rule A: If the item type comparison result is inconsistent, generate an update instruction for the Type field, with the new value being Sensed_Item_Type; Rule B: If the relative difference percentage of the quantity / fill level comparison is greater than a preset threshold (e.g., 5%), generate an update instruction for the Current_Qty field, with the new value being Sensed_Quantity_Value; Rule C: If the internal state comparison shows that the state has changed from "normal" to "abnormal", generate an update instruction for the State field, with the new value being the status code of Sensed_Internal_State, and generate an insert instruction into the Inventory_State_History table to record the detailed exception description and the sensed timestamp; Rule D: Regardless of whether there are differences, generate an update instruction for the Last_Inventory_Time field, with the new value being Sensing_Timestamp. The system traverses the data difference analysis table, applies the corresponding update rules, and constructs one or more database operation instructions (e.g., SQL UPDATE or INSERT statement objects). These instructions, along with the Item_ID and Location_ID they target, and the source information of this update (such as the report ID), are encapsulated into an update data packet.

[0129] Evaluate the permissions and priorities of each database operation instruction in the update data packet according to a preset update policy matrix. The update policy matrix defines the processing methods and priorities of different types of update operations (e.g., modifying quantity, modifying status, modifying type) under different conditions (e.g., magnitude of quantity difference, severity of status abnormality, item criticality, etc.). For example, the policy matrix stipulates that: ① If the quantity adjustment is less than 5% and the sensed reliability score is greater than 0.8, it is marked as "auto-execute" with a "low" priority; ② If the quantity adjustment is greater than 20% or the item type does not match, it is marked as "requiring manual review" with a "medium" priority; ③ If the internal state changes to "severely abnormal" (e.g., a liquid container is damaged), it is marked as "auto-execute" with a "high" priority and triggers a subsequent alarm process; ④ If the internal state changes to "slightly abnormal" (e.g., suspected internal looseness), it is marked as "auto-execute" with a "medium" priority and marks the item for manual inspection when it is next shipped out. The system traverses each instruction in the update data packet, refers to the update policy matrix, and assigns an execution permission (e.g., "AUTO_EXECUTE" or "MANUAL_REVIEW") and a priority (e.g., levels 1-5, with 1 being the highest) to it. Attach this permission and priority information to each instruction in the update data packet to form a hierarchical update plan.

[0130] Output all database operation instructions marked as "AUTO_EXECUTE" in the hierarchical update plan. The system starts a database transaction. Inside the transaction, the system executes these automatically executed database operation instructions in order of priority. For example, for an UPDATE statement that updates the Inventory_Main table and an INSERT statement that inserts into the Inventory_State_History table, the system submits these statements through the database connection pool. The database management system is responsible for executing these operations atomically: if all statements are successfully executed, the transaction is committed (COMMIT), and all changes are permanently saved; if any error occurs during execution (e.g., database connection interruption, data conflict), the transaction is rolled back (ROLLBACK), and all changes made in this transaction are undone, and the database is restored to the state before the transaction started. For instructions marked as "MANUAL_REVIEW", the system does not execute them at this stage but generates a pending task and routes it to the manual review workflow. The system records the execution result (success / failure), execution time, item identifiers involved, and update type of each transaction, as well as any error messages that occur. These execution records and transaction status information constitute the database update result.

[0131] Preferably, the business process trigger determination in step S4 includes:

[0132] Perform update result classification and parsing on the database update result to obtain a classified update record table;

[0133] Match the exception handling rules against the classified update record table to obtain an exception handling solution table;

[0134] Generate an item relocation instruction set based on the exception handling solution table;

[0135] Generate an inventory review task table based on the exception handling solution table;

[0136] Adjust the outbound strategy for quality change records in the classified update record table to obtain an outbound rule modification instruction;

[0137] Generate business process instructions based on the item relocation instruction set, inventory review task table, and outbound rule modification instruction;

[0138] Integrate the business process trigger instruction and the database update result to generate a warehousing status update instruction.

[0139] In the embodiments of the present invention, these update results are parsed to identify which fields of which items have changed and what type of update (e.g., quantity correction, status change, type mismatch) the change is triggered by the sensed data. These information are organized into a classified update record table. Based on the detailed change information in the classified update record table, the system checks against a preset exception handling rule library to find and match the subsequent business processes that need to be triggered, and obtains an exception handling solution table. Based on the action items determined in the exception handling solution table, the system generates an item relocation instruction set for moving the exception items to the designated area. At the same time, for situations that require manual intervention for verification, the system generates an inventory review task table for arranging personnel to conduct on-site inspections or re-inventory. For the quality changes or status anomalies of items revealed by the sensed data, the system generates an outbound rule modification instruction to adjust the priority or availability of the item in the outbound picking process. The system integrates the item relocation instruction set, the inventory review task table, and the outbound rule modification instruction into one or more business process instruction data packets. Finally, the system merges these business process instruction data packets with the original database update results to form a complete warehousing status update instruction containing the details of inventory changes and subsequent processing instructions.

[0140] Traverse the database update result records. For the UPDATE operation on the Inventory_Main table, the system checks the modified fields and identifies the update type: if the Current_Qty field is modified, it is marked as "quantity change"; if the State field is modified, it is marked as "status change"; if the Item_Type field is modified, it is marked as "type change"; if the Last_Inventory_Time field is modified, it is marked as "inventory time update". For the INSERT operation on the Inventory_State_History table, it is marked as "new abnormal status record", and the detailed abnormal description and sensed reliability score are extracted from the inserted data. The system organizes the parsed information, including item identification, location identification, update type, specific change content (e.g., quantity change value, new status code), and the related sensed reliability score, etc., into a structured table to form a classified update record table.

[0141] The system internally stores an exception handling rule library. For example, the rule : If the update type is "quantity change" and the absolute quantity difference exceeds 10 or the relative difference exceeds 15% and the sensed reliability score is greater than 0.7, then trigger the action "create an inventory review task". The rule : If the update type is "status change" and the new status is marked as "severe exception" (e.g., code 901 indicates "liquid container damaged"), then trigger the actions "generate relocation instruction to the quality inspection area" and "adjust the outbound strategy to prohibit outbound". Rule R3: If the update type is "type change" and the perception type confidence is lower than 0.85, then trigger the action "create type verification task". The system traverses each record in the classification update record table. For each record, the system evaluates all the rule conditions in the exception handling rule library. If a record meets the conditions of a certain rule, the system adds the actions defined by that rule (e.g., "create inventory review task", "generate relocation instruction", "adjust outbound strategy") to the exception handling plan corresponding to that record. If a record meets multiple rules, the corresponding actions are accumulated. The system summarizes all item identifiers and their corresponding lists of actions to be performed to form an exception handling plan table.

[0142] Traverse the exception handling plan table to find records containing the action "generate relocation instruction". For each such record, the system determines the target storage location (e.g., pre-set quality inspection area location, scrap area location, pending processing area location) to which the item needs to be relocated according to the exception type or rule definition. The system generates the detailed robot instructions required to perform this relocation operation by querying the warehouse layout and robot capacity information. The instruction structure includes: task type ("relocation"), item identifier (Item_ID), current location (Location_ID), target location (Target_Location_ID), reason for relocation (e.g., "severe internal exception detected"), priority (e.g., high priority). The system collects all the items that need to be relocated and their corresponding relocation instructions to form an item relocation instruction set. These instructions will be sent to the warehouse execution system (WES) or directly to the robots responsible for handling for execution.

[0143] Traverse the exception handling plan table to find records containing the action "create inventory review task". For each such record, the system generates a review task that requires human or dedicated equipment intervention according to the triggering reason (e.g., large quantity difference, type in doubt, low perception reliability). The task structure includes: task type ("inventory review"), item identifier (Item_ID), location identifier (Location_ID), review content (e.g., "verify quantity", "verify item type", "check internal status"), description of triggering reason (e.g., "the perceived estimated quantity does not match the system record, with a difference of 18%"), task priority, recommended review method (e.g., manual count, use of specific detection equipment). The system collects all the items that need to be reviewed and their corresponding review task information to form an inventory review task table. These tasks will be sent to the warehouse operation management system or the manual task assignment system.

[0144] Filter out the records in the table related to the quality or status change of the items, such as records with the update type being "status change", the new status marked as abnormal, or a large quantity change indicating damaged packaging. An outbound strategy management module is maintained internally in the system, which determines the priority and availability of items during picking based on the status, batch, attributes, etc. of the items. The system generates an instruction to modify the outbound rules of the item according to the filtered quality change records. The instruction structure includes: item identifier (Item_ID), location identifier (Location_ID), new outbound status flag (e.g., "normal", "priority outbound", "delayed outbound", "forbidden outbound"), and reason for adjustment (e.g., "slight abnormality detected, priority outbound recommended", "severe abnormality detected, forbidden outbound"). For example, if a slight shaking abnormality (but not damaged) is detected in a liquid container, the system generates an instruction to increase the outbound priority of this batch of items for quick consumption. If a severe damage is detected, an instruction is generated to mark the item as forbidden outbound. The system collects all the items that need to adjust the outbound strategy and their corresponding modification instructions to form an outbound rule modification instruction. These instructions will be sent to the outbound picking system.

[0145] Integrate the instructions and tasks for different downstream systems to generate one or more unified business process instruction data packets. This data packet is a structured message that contains all the subsequent business process action items triggered by this perceived inventory check. For example, the data packet can contain different parts: a list for storing all the stock transfer instructions, a list for storing all the recheck tasks, and a list for storing all the outbound rule modification instructions. The data packet also contains a total task identifier, trigger time, and associated perceived report identifier for tracking. This integrated data structure is the business process instruction. It transforms the perceived results into specific warehouse operation and management tasks.

[0146] The warehouse status update instruction is one of the final outputs of this perceived inventory check cycle. It not only confirms the update results of the inventory master data (which items have quantity, status, etc. changed), but also clarifies all the subsequent business process actions triggered thereby (which items need stock transfer, which need recheck, and which outbound strategies are adjusted). The instruction structure includes: unique instruction identifier; associated inventory check task identifier and perceived report identifier; database update details (list of items successfully updated, updated fields and new values); triggered business process details (list of stock transfer instructions, list of recheck tasks, list of outbound rule modification instructions); and other meta information (such as generation time, execution system identifier). This comprehensive instruction is sent to the relevant warehouse management subsystems (e.g., WES, TMS, WCS) and monitoring interfaces for subsequent execution, tracking, and manual intervention.

[0147] Therefore, in any regard, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Thus, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be embraced within the present invention.

[0148] The above description is only a specific implementation manner of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A warehousing inventory method, characterized in that, It includes the following steps: Step S1: Analyze the inventory taking demand path of the warehouse to obtain an inventory taking path plan; issue inventory taking tasks to the robot according to the inventory taking path plan to obtain handling task instructions; Step S2: Perform grasping processing on the inventory items according to the handling task instructions to obtain static contact characteristic data; perform dynamic interaction and multi-modal signal acquisition based on the static contact characteristic data to obtain interaction perception data; Step S3: Separate the perception signals of the interaction perception data to obtain a mechanical parameter feature table and an acoustic feature map; Perform item type recognition and matching on the mechanical parameter feature table and the acoustic feature map to obtain an item type determination result; perform an internal state evaluation of the items based on the item type determination result to obtain an internal state evaluation table; estimate the quantity / filling amount and generate a perception report for the items identified as container types in the item type determination result according to the internal state evaluation table to obtain a material perception feature report; Step S4: Synchronously update the inventory information based on the material perception feature report to obtain a database update result; perform a business process trigger judgment based on the database update result to obtain a warehouse status update instruction.

2. The warehousing inventory method according to claim 1, characterized in that Step S1 includes the following steps: Step S11: Obtain the inventory status data of the current warehouse and generate an inventory taking demand list according to the preset inventory taking rules; Step S12: Plan the operation path of the robot according to the inventory taking demand list to obtain an inventory taking path plan; Step S13: Obtain the robot ability data and item characteristic data, and perform robot ability matching according to the inventory taking path plan to obtain a robot allocation result; Step S14: Configure the perception actions of the robot according to the robot allocation result to obtain a perception action configuration table; Step S15: Package and issue instructions for the inventory taking path plan, robot allocation result, and perception action configuration table to obtain handling task instructions.

3. A warehousing inventory method according to claim 1, characterized in that, The inventory item grasping process in Step S2 includes: Parse the instruction of the handling task instruction and pre-activate the sensor to obtain a sensor configuration plan; Prepare for approaching and positioning the items in the storage location according to the sensor configuration plan and the storage location information in the handling task instruction to obtain a grasping execution plan; Perform initial contact and basic characteristic acquisition according to the grasping execution plan to obtain static contact characteristic data.

4. A warehousing inventory method according to claim 1, characterized in that The dynamic interaction and multi-modal signal acquisition in Step S2 includes: Control the increase of the grasping force based on the static contact characteristic data to obtain a grasping process characteristic curve; Analyze the lifting gravity response according to the grasping process characteristic curve to obtain a lifting gravity response curve; Evaluate the suspension stability of the item according to the lifting gravity response curve to obtain suspension stability data; Perform perception action execution based on the suspension stability data and enhance the micro-perturbation to obtain perturbation action response data; Collect sensor array data according to the perturbation action response data; Perform multi-modal data time alignment on the sensor array data to obtain a time-synchronized data stream; Integrate the perturbation action response data and the time-synchronized data stream to obtain dynamic interaction response data; Synchronize and denoise the signals of the dynamic interaction response data to obtain a denoised signal data packet; Perform data encapsulation and transmission on the noise-reduced signal data packet to obtain interactive perception data.

5. A warehousing inventory method according to claim 1, characterized in that The perception signal separation in step S3 includes: Perform signal separation and preprocessing on the interactive perception data to obtain a classification signal feature set; Extract mechanical characteristic parameters from the classification signal feature set to obtain a mechanical parameter feature table; Generate an acoustic feature spectrogram from the classification signal feature set to obtain an acoustic feature spectrogram.

6. A warehousing inventory method according to claim 5, characterized in that The generation of the acoustic feature spectrogram includes: Perform acoustic and vibration time-frequency conversion on the classification signal feature set to obtain a time-frequency energy matrix; Based on the time-frequency energy matrix, perform material feature frequency band division to obtain a frequency band statistical feature table; Perform spectral centroid and morphology calculations on the time-frequency energy matrix to obtain a spectral morphology parameter sequence; Perform time-varying analysis of the acoustic energy envelope on the classification signal feature set to obtain an envelope dynamic parameter table; Perform acoustic fingerprint matching calculations on the frequency band statistical feature table, the spectral morphology parameter sequence, and the envelope dynamic parameter table to obtain a material acoustic feature matching table; Based on the material acoustic feature matching table and the envelope dynamic parameter table, extract the filling rate acoustic markers to obtain a filling state acoustic index table; Perform abnormal resonance and acoustic deviation detection on the time-frequency energy matrix to obtain an abnormal acoustic feature table; Generate an acoustic feature spectrogram according to the abnormal acoustic feature table and the filling state acoustic index table.

7. A warehousing inventory method according to claim 1, characterized in that, The quantity / filling amount estimation and perception report generation in step S3 include: Extract the filling-related parameter set from the items identified as container types in the item type determination result; Match the filling-related parameter set with the pre-established physical model parameter library to obtain an estimation model selection result; Based on the estimation model selection result, calculate the liquid filling amount to obtain the calculated value of the liquid filling rate; Based on the estimation model selection result, estimate the particle quantity to obtain the calculated value of the particle filling rate; Based on the estimation model selection result, estimate the solid item count to obtain the estimated value of the item quantity; Perform an estimation reliability score on the calculated value of the liquid filling rate, the calculated value of the particle filling rate, and the estimated value of the item quantity to obtain an estimation reliability score; Calibrate the estimation reliability score with historical data and generate a filling amount estimation result; Generate a comprehensive report on the item type determination result, the internal state evaluation table, and the filling amount estimation result to obtain a material perception feature report.

8. A warehousing inventory method according to claim 1, characterized in that, The inventory information synchronization and update in step S4 include: Receive and verify the material perception feature report to obtain a verification result record; Compare the material perception feature report with the inventory data according to the verification result record to obtain a data difference analysis table; Update the inventory record according to the data difference analysis table to obtain an update data packet; Evaluate the update permission and priority of the update data packet to obtain a hierarchical update plan; Execute the database transaction according to the hierarchical update plan to obtain a database update result.

9. A warehousing inventory method according to claim 1, characterized in that The business process trigger judgment in step S4 includes: Analyze the update result classification of the database update result to obtain a classification update record table; Match the abnormal handling rules with the classification update record table to obtain an abnormal handling plan table; Generate an item relocation instruction set based on the abnormal handling plan table; Generate a physical inventory review task table based on the abnormal handling plan table; Adjust the outbound strategy for the quality change records in the classification update record table to obtain an outbound rule modification instruction; Generate a business process instruction based on the item relocation instruction set, the inventory check re-verification task table, and the outbound rule modification instruction; Integrate the business process trigger instruction and the database update result to generate a warehousing status update instruction.

10. A warehousing system, applied to a warehousing inventory method described in any one of claims 1-9, characterized in that, This warehousing system includes: An inventory check task assignment module, which is used to analyze the inventory check requirement path of the warehouse to obtain an inventory check path plan; issue inventory check tasks to the robot according to the inventory check path plan to obtain handling task instructions; An item interaction perception acquisition module, which is used to perform inventory item grasping processing according to the handling task instructions to obtain static contact characteristic data; perform dynamic interaction and multi-modal signal acquisition based on the static contact characteristic data to obtain interaction perception data; A material feature intelligent analysis module, which is used to separate the perception signals of the interaction perception data to obtain a mechanical parameter feature table and an acoustic feature map; perform item type recognition and matching on the mechanical parameter feature table and the acoustic feature map to obtain an item type determination result; perform an internal state assessment of the item based on the item type determination result to obtain an internal state assessment table; estimate the quantity / filling amount and generate a perception report for the items identified as container types in the item type determination result according to the internal state assessment table to obtain a material perception feature report; An inventory status update and trigger module, which is used to synchronously update the inventory information of the material perception feature report to obtain a database update result; perform a business process trigger judgment based on the database update result to obtain a warehousing status update instruction.

Images