Warehouse inventory method and warehouse system

Abstract

The present invention relates to the field of intelligent warehouse management and Internet of Things perception technology, and in particular to a warehouse inventory method and a warehouse system. The method comprises the following steps: performing inventory demand path analysis on the warehouse to obtain an inventory path plan; issuing inventory tasks to the robot according to the inventory path plan to obtain handling task instructions; performing inventory item grabbing and processing according to the handling task instructions to obtain static contact characteristic data; performing dynamic interaction and multimodal signal acquisition based on the static contact characteristic data to obtain interactive perception data; performing perception signal separation on the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature map; performing item type identification and matching on the mechanical parameter feature table and the acoustic feature map to obtain an item type determination result. The present invention improves the dimension and accuracy of inventory information by seamlessly integrating the inventory function into the daily operation process, thereby greatly improving the overall efficiency of warehouse operations.

Description

Technical Field

[0001] The present invention relates to the field of intelligent warehouse management and Internet of Things perception technology, and in particular to a warehouse inventory method and a warehouse system. Background Art

[0002] Existing warehouse inventory counting technology primarily focuses on external identification and location confirmation of items, but its ability to perceive their internal condition is severely limited. When items are intact but damaged or deteriorating, traditional inventory counting methods fail to detect these hidden issues, resulting in significant discrepancies between inventory information and actual conditions, which in turn impacts the accuracy of downstream business decisions. For specialized materials such as liquids and granules, traditional inventory counting methods struggle to determine the actual fill level or contents without opening or intervening. Inventory counting of these materials often relies on manual spot checks or rough estimates, which is not only inefficient but also prone to subjective errors and fails to meet the demands of modern, refined warehouse management. Traditional inventory counting processes are separated from daily operations, requiring dedicated time and manpower for inventory counting. This fragmented management model not only incurs additional labor costs but also disrupts the normal pace of warehouse operations, reducing overall efficiency. Especially for large warehouses, a complete inventory count cycle can take days or even weeks, making it difficult to ensure the real-time nature of inventory data.

[0003] In summary, existing technologies have problems such as the inability to perceive the internal state of items, difficulty in accurately counting special materials, and the separation of inventory from daily operations, resulting in low efficiency, which need to be solved urgently. Summary of the Invention

[0004] Based on this, it is necessary to provide a warehouse inventory method and a warehouse system to solve at least one of the above technical problems.

[0005] To achieve the above purpose, a warehouse inventory method includes the following steps:

[0006] Step S1: Analyze the inventory demand path of the warehouse to obtain the inventory path plan; issue the inventory task to the robot according to the inventory path plan to obtain the handling task instruction;

[0007] Step S2: performing inventory item grabbing processing according to the handling task instruction to obtain static contact characteristic data; performing dynamic interaction and multimodal signal collection based on the static contact characteristic data to obtain interactive perception data;

[0008] Step S3: Performing perception signal separation on the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature spectrum; performing item type identification and matching on the mechanical parameter feature table and the acoustic feature spectrum to obtain an item type determination result; performing an item internal state assessment based on the item type determination result to obtain an internal state assessment table; and performing quantity / filling amount estimation and perception report generation on items identified as containers in the item type determination result based on the internal state assessment table to obtain a material perception feature report.

[0009] Step S4: Synchronously update the inventory information of the material perception feature report to obtain a database update result; perform business process triggering judgment based on the database update result to obtain a warehouse status update instruction.

[0010] This invention achieves automated and targeted inventory task generation by intelligently analyzing warehouse inventory status and automatically screening items for inventory based on preset rules. By combining warehouse layout and robot capabilities, the system calculates the optimal work path and robot dispatch plan, improving overall inventory efficiency and resource utilization. Specifically, the system finely configures sensing action parameters and sensor operating modes based on item characteristics, ensuring targeted and high-quality subsequent data collection. This lays the foundation for perceiving items' internal states through physical interaction, overcoming the limitations of traditional inventory task issuance, which focuses solely on location and identification. This system seamlessly integrates the perception of items' internal states and physical properties into routine automated handling operations. Through controlled slow approach, initial contact detection, constant force maintenance, grasping, and lifting, along with basic robot operations, the system utilizes a high-precision multimodal sensor array for real-time, high-frequency data acquisition, capturing the mechanical and acoustic responses of items under varying load conditions. Furthermore, by performing object-specific sensing actions (such as shaking and squeezing) and superimposing micro-perturbations, key characteristic signals of the items' internal material states and structures are generated. Precise time alignment and noise reduction of multimodal data ensure the synchronization and purity of collected data, providing high-quality raw input for subsequent accurate analysis of hidden information about items and significantly improving the dimensionality of inventory data. Deep intelligent analysis of collected multimodal interactive perception data enables non-invasive, high-dimensional assessment of item type, internal state, and quantity / fill level. The system extracts quantitative features reflecting an item's physical properties, internal structural stability, and contents from force / torque, vibration, and acoustic signals, and compares these features against a database of known item features. This not only improves item identification accuracy but also detects internal damage, loose components, clumping of particles, or abnormal liquid sloshing even in otherwise intact packaging. For materials such as liquids and particles, which are traditionally difficult to accurately count, analysis of their specific dynamic responses and acoustic fingerprints assists in estimating the container's fill level or approximate quantity, significantly improving the efficiency and precision of inventory counting for specialized materials. The resulting perception report provides rich and accurate information on material status for downstream inventory management and business decision-making. Deep integration of the material perception feature reports derived from intelligent analysis with warehouse inventory management systems ensures real-time, synchronized updates of inventory information. The system automatically compares perceived data with existing inventory records and, based on pre-set rules and assessed reliability, automatically or after manual review, adjusts inventory quantities, updates the item's internal status, and updates the last count time. More importantly, based on perceived anomalies (such as significant quantity discrepancies or abnormal internal status), the system automatically triggers appropriate business processes. These include generating transfer instructions to move the anomalous item to a waiting area, creating an inventory review task requiring manual verification, and adjusting the item's outbound picking policy (e.g., prohibiting or prioritizing outbound delivery).This automated, rule-driven exception handling process significantly improves the warehouse's response speed and processing efficiency to abnormal situations, ensuring the accuracy of inventory data and the smooth progress of business processes.

[0011] Therefore, this invention provides a warehouse inventory method that integrates a multimodal sensor system (force / torque, vibration, and acoustics) into an automated handling robot to simultaneously collect physical interaction response data from items during daily handling operations. Leveraging advanced signal processing and machine learning algorithms, this data is used to extract the internal state characteristics of items, enabling non-invasive detection of issues such as internal damage and abnormal fill levels. This method seamlessly integrates inventory counting into daily operational processes, significantly improving the dimensionality and accuracy of inventory information and enabling coordinated inventory counting and operational operations. This significantly enhances the overall efficiency of warehouse operations and provides a new technical approach for modern, refined warehouse management.

[0012] Preferably, the present invention further provides a warehousing system for executing the above-mentioned warehousing inventory counting method, the warehousing system comprising:

[0013] The inventory task dispatching module is used to analyze the inventory demand path of the warehouse and obtain the inventory path plan; according to the inventory path plan, the inventory task is issued to the robot to obtain the handling task instruction;

[0014] The item interaction perception and acquisition module is used to process inventory items according to handling task instructions to obtain static contact characteristic data; based on the static contact characteristic data, dynamic interaction and multimodal signal acquisition are performed to obtain interaction perception data;

[0015] The material feature intelligent analysis module is used to separate the perception signals of the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature spectrum; perform item type identification and matching on the mechanical parameter feature table and the acoustic feature spectrum to obtain an item type determination result; perform an internal status assessment of the item based on the item type determination result to obtain an internal status assessment table; and perform quantity / filling volume estimation and perception report generation for items identified as containers in the item type determination result based on the internal status assessment table to obtain a material perception feature report;

[0016] The inventory status update and trigger module is used to synchronize the inventory information of the material perception feature report to obtain the database update result; based on the database update result, the business process trigger judgment is performed to obtain the warehouse status update instruction.

[0017] This warehouse system integrates an inventory task dispatching module, an item interaction sensing and collection module, a material feature intelligent analysis module, and an inventory status update and triggering module to achieve automated, intelligent, and efficient warehouse inventory management. The inventory task dispatching module intelligently generates and optimizes inventory tasks based on warehouse status and pre-set rules, automatically planning robot paths and configuring sensing actions, improving the efficiency and targeted nature of task planning. The item interaction sensing and collection module simultaneously collects multimodal data using high-precision force / torque, vibration, and acoustic sensors during automated robotic handling. By leveraging controlled physical interactions to stimulate internal item feature responses, this module achieves non-invasive sensing of an item's internal state and physical properties, overcoming the limitations of traditional methods that rely solely on external identification. The material feature intelligent analysis module performs in-depth processing and analysis of the complex sensory data collected, accurately identifying item type and assessing internal conditions (such as detecting hidden damage, looseness, and agglomeration). It also innovatively assists in estimating the quantity or fill level of difficult-to-accurate materials such as liquids and granules, significantly enriching inventory information and improving data accuracy. The inventory status update and triggering module synchronizes perception and analysis results to the inventory database in real time, ensuring that inventory records are consistent with the actual status of items. It also automatically triggers subsequent business processes such as inventory transfers, rechecks, and adjustments to outbound delivery strategies based on perceived anomalies (such as quantity discrepancies or abnormal status), improving the response speed of exception handling and overall warehouse operational efficiency. In summary, this warehousing system seamlessly integrates inventory counting into daily operations, providing a more sophisticated, accurate, and efficient inventory management method than traditional methods, significantly improving the level of automation and information quality in warehouse management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The figure is a flowchart of the steps of a warehouse inventory method.

[0019] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0020] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. It is obvious that the embodiments described are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of the present invention.

[0021] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller approaches.

[0022] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0023] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a flow chart showing the steps of the warehouse inventory counting method of the present invention. In this example, the warehouse inventory counting method includes the following steps:

[0024] Step S1: Analyze the inventory demand path of the warehouse to obtain the inventory path plan; issue the inventory task to the robot according to the inventory path plan to obtain the handling task instruction;

[0025] In the embodiment of the present invention, the current inventory status is obtained by querying the database, and the list of items to be counted is screened according to the preset inventory rules (such as based on time, location, inventory or abnormality mark). The system uses the warehouse layout and the real-time position of the robot, combined with the path planning algorithm (such as Using a similar algorithm to the traveling salesman problem (TSP) algorithm, the system calculates the optimal, ordered path to the inventory locations of the items to be counted, generating a count route plan. The system obtains the detailed physical attributes of the items to be counted (weight, packaging, contents) and the operational capabilities of available robots (payload, gripper, sensor configuration). Using a matching scoring mechanism and an assignment algorithm, the system determines the most suitable robot for the task and generates a robot assignment. Based on the robot assignment results and the item characteristics (material, contents), the system queries a sensor action library, matches action templates, and calculates precise sensor action parameters (such as sway amplitude, frequency, and compression pressure / deformation) based on the specific item / robot parameters. These calculated parameters are adjusted based on item and robot safety limits to form a safety-constrained action plan. Sensors are then configured (sampling rate, sensitivity) to generate a sensor configuration plan. Finally, the system integrates the count route plan, robot assignment results, safety-constrained action plan, and sensor configuration plan into a structured handling task instruction that includes item identification, location, task type, execution sequence, sensor action parameters, and exception handling plans. This instruction is then delivered to the designated robot execution terminal via a secure communication channel.

[0026] Step S2: performing inventory item grabbing processing according to the handling task instruction to obtain static contact characteristic data; performing dynamic interaction and multimodal signal collection based on the static contact characteristic data to obtain interactive perception data;

[0027] In this embodiment of the present invention, after receiving a handling task instruction, the robot analyzes the instruction content and configures the operating parameters (sampling frequency and sensitivity) of the onboard multimodal sensors (force / torque sensor, vibration sensor, and micro-microphone) based on the specified object type and sensing action requirements, completing sensor pre-activation. The robot then navigates to the target storage area, uses the vision system to accurately locate the object's position and posture, calculates the optimal grasping point, grasping posture, and approach trajectory, and generates a grasping execution plan. The robot slowly approaches the object along the planned trajectory, monitoring force sensor data in real time. When the force detected in the approach direction exceeds a preset threshold (e.g., 0.5N), initial contact is determined and the sensor and robot posture data at that moment are recorded as contact trigger data. The robot then switches to force control mode, maintaining a constant low contact force (e.g., 2N) for a short period of time based on the contact parameters. During this period, the robot collects sensor data and performs multi-point micro-deformation testing to obtain static contact characteristic data reflecting the object's surface and initial mechanical properties. Next, the robot gradually increases its gripping force to the target value according to the grasping plan. Once the grasp is secure, it lifts the object into a suspended state. During this process, force / torque data is continuously collected to generate a grasping process characteristic curve and a lift-to-gravity response curve. The system analyzes the lift-to-gravity response curve to assess the object's stability while suspended. With the object stable, the robot executes the sensing actions (shaking, squeezing) specified in the safety constraint action plan and superimposes micro-perturbations. The sensor array synchronously collects high-frequency response data to generate perturbation action response data. The system performs high-precision time alignment on the collected multimodal sensor array data to generate a time-synchronized data stream. The system then integrates the data from all dynamic stages (curves, stability data, perturbation responses, and synchronization data). Finally, the system performs signal synchronization verification and noise reduction (digital filtering and spectral subtraction) on the integrated dynamic interaction response data to generate a de-noised signal data packet. This signal packet is then packaged with metadata such as the object's identification and transmitted to the central processing system via a secure channel to form interactive perception data.

[0028] Step S3: Performing perception signal separation on the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature spectrum; performing item type identification and matching on the mechanical parameter feature table and the acoustic feature spectrum to obtain an item type determination result; performing an item internal state assessment based on the item type determination result to obtain an internal state assessment table; and performing quantity / filling amount estimation and perception report generation on items identified as containers in the item type determination result based on the internal state assessment table to obtain a material perception feature report.

[0029] In this embodiment of the present invention, the force / torque, vibration, and acoustic signal data are separated and subjected to modal-specific preprocessing (low-pass filtering, bandpass filtering, and short-time Fourier transform) to obtain a set of classified signal features. The system extracts parameters reflecting the object's mechanical properties, such as mass, stiffness, elasticity, and damping, from the force / torque and vibration signals to generate a mechanical parameter feature table. The system further performs time-to-frequency transformation (STFT) on the acoustic and vibration signals to generate a time-frequency energy matrix. Based on this time-frequency energy matrix, the system calculates the energy statistics of different material characteristic frequency bands and extracts spectral morphological parameters (such as spectral centroid and spread) for each time frame. Simultaneously, the system analyzes the acoustic signal's energy envelope to extract its dynamic parameters (such as duration and decay time). The system integrates these frequency band statistics, spectral morphology, and envelope dynamic features to form a high-dimensional acoustic feature vector. This vector is then pattern-matched (e.g., using Euclidean distance and similarity calculations) against a pre-defined acoustic fingerprint database to generate a material acoustic feature matching table, enabling preliminary object identification. Based on the identified container item type and acoustic characteristics (particularly sloshing energy, decay time, and particle energy), the system applies a pre-calibrated mapping model or regression formula to estimate the fill rate of liquids or the number of granular or solid items, and incorporates mechanical parameters (such as estimated mass and moment of inertia) for auxiliary verification. The system calculates a reliability score for the estimated result (based on signal-to-noise ratio, recognition confidence, modal consistency, etc.) and calibrates it using historical data to generate a fill quantity estimate that includes the estimated value, error range, and reliability score. Simultaneously, the system compares the current time-frequency energy matrix with a normal item template to detect abnormal resonances, acoustic deviations, or transient noise, and generates an abnormal acoustic signature table. Ultimately, the system integrates the item type determination results, internal state assessment (based on abnormal acoustic / mechanical signature determination), and fill quantity estimation results into a structured material perception feature report that comprehensively describes the item's perceived state.

[0030] Step S4: Synchronously update the inventory information of the material sensing feature report to obtain a database update result; perform business process triggering judgment based on the database update result to obtain a warehouse status update instruction;

[0031] In this embodiment of the present invention, format, field, and identifier validity verification is first performed, generating a verification result record. For reports that pass verification, the system queries the inventory master table to obtain the current item record. The system then compares the item type, quantity / fill level, and internal status in the perception report with the inventory record in detail, calculates the difference values ​​and percentages, marks the status change, and generates a data difference analysis table. Based on the data difference analysis table and pre-set update rules (for example, updating the quantity if the quantity difference exceeds a threshold, or updating the status if the status becomes abnormal), the system generates database operation instructions (such as SQL UPDATE / INSERT statements) to modify the inventory database, forming an update data packet. The system compares the update policy matrix to evaluate the permission requirements (automatic execution or manual review) and business priority (high, medium, low) of each instruction in the update data packet, and generates a hierarchical update plan. The system filters out instructions marked for automatic execution, initiates a database transaction, and executes these instructions in order of priority. The transaction execution results (success / failure) and detailed logs are recorded to form the database update results. The system parses the database update results, identifies the actual item records that have changed and the type of change (quantity change, status change, type change), and generates a categorized update record table. The system compares the exception handling rule base and matches the information in the classification update record table with the subsequent business process actions that need to be triggered (such as inventory transfer, review, and adjustment of outbound delivery policies), thereby generating an exception handling plan table. Based on this exception handling plan table, the system generates a specific set of item transfer instructions (specifying the target location), an inventory review task table (specifying the review content and method), and outbound delivery rule modification instructions (specifying the new outbound delivery status or priority). Ultimately, the system integrates these business process instructions (inventory transfer, review, outbound delivery adjustment) with the database update results to form a complete warehouse status update instruction containing inventory change details and subsequent action instructions, which is then sent to the relevant warehouse execution and management systems.

[0032] Preferably, step S1 includes:

[0033] Step S11: Obtain the current warehouse inventory status data and generate an inventory requirement list according to the preset inventory rules;

[0034] Step S12: planning the robot's operation path according to the inventory requirement list to obtain an inventory path plan;

[0035] Step S13: Obtain robot capability data and item characteristic data, and perform robot capability matching according to the inventory path plan to obtain a robot allocation result;

[0036] Step S14: configuring the robot's perception action according to the robot allocation result to obtain a perception action configuration table;

[0037] Step S15: Encapsulate and issue the inventory path plan, robot allocation results, and perception action configuration table to obtain the handling task instructions.

[0038] In an embodiment of the present invention, by calling a database interface (for example, using an SQL query statement), all item records currently stored in the warehouse are obtained from the back-end inventory main table (Inventory_Main) and inventory history table (Inventory_History). The inventory main table contains at least the following fields: item ID (Item_ID), current location (Location_ID), current inventory quantity (Current_Qty), and inbound time (Inbound_Time). The inventory history table contains: item ID (Item_ID), inventory time (Inventory_Time), inventory quantity (Inventory_Qty), and inventory result status (Status). After receiving this raw inventory status data, the system loads it into memory for processing. The system internally stores a preset inventory rule set that defines the conditions for triggering inventory tasks. For example, a rule set might include the following: Rule 1: The last inventory count of an item is more than 90 days old; Rule 2: The item's current location falls within the designated storage area for this planned regional inventory count; Rule 3: The difference between the item's current inventory quantity and the safety stock threshold (Safety_Stock) is less than -10% (indicating a critical inventory shortage); and Rule 4: Items that have been flagged by the system as having abnormal quantity or status during other operations (such as outbound verification). The system traverses each acquired inventory record and matches it against the rule set. If a record meets any of the rules, the item's unique identifier (Item_ID) and current location information (Location_ID) are extracted and added to a temporary list. After traversing all records, this temporary list is organized and named the inventory request list, which is an ordered or unordered collection of the item identifiers and location information to be counted.

[0039] By querying the warehouse basic data, the system converts each location ID into its corresponding three-dimensional space coordinates ( , , ). The system also obtains the real-time location information of the currently available robots ( , , ) and the global map data of the warehouse. The global map data is represented by a graph structure, where nodes represent channel intersections or storage locations, and edges represent channels, including edge weights (such as walking distance and travel time). The operation path planning module uses the A algorithm (or other path search algorithms, such as the Dijkstra algorithm) to calculate the robot's current position ( , , ) as the starting point, take all the storage location coordinates in the inventory demand list {( , , ),( , , ),...,( , , )} is the set of points that must be visited, and the location (x_charge, y_charge, z_charge) at which the robot returns to the charging station is the end point. Calculate a path sequence with the lowest total path cost (for example, the shortest total walking distance). In order to optimize the visit sequence, an approximate algorithm for the Traveling Salesman Problem (TSP) can be combined with the A algorithm to determine the number of visits {( , , ),...,( , , The algorithm outputs an ordered sequence of location visits, for example: Loc_A1 → Loc_C7 → Loc_B3 → ... → Loc_Charge. This ordered sequence of location visits, along with the robot's path from its current position to the first location in the sequence, constitutes the inventory routing plan.

[0040] For each item in the plan, the system queries the item characteristics database to obtain detailed physical properties, such as the item's nominal weight (Weight_Nominal, in kg), packaging type (e.g., carton, plastic barrel, metal can), and whether it contains special media markings such as liquids, granules, or fragile parts. Simultaneously, the system queries the capabilities list of all available robots in the system. This robot capabilities list contains each robot's unique identifier (Robot_ID) and its operational capability parameters, such as its maximum gripping load (Max_Load_Capacity, in kg), gripper type (Gripper_Type, e.g., gripper, fork arm, suction cup), integrated sensor type (e.g., force sensor, vibration sensor, microphone), and sensor accuracy and sampling rate. For each item to be counted in the inventory route plan, the system evaluates the compatibility between its physical properties and the available robot capabilities. This compatibility assessment utilizes a scoring mechanism, such as constructing a matching function called Score(Item_Properties, Robot_Capabilities). This function considers multiple dimensions: if the item weight Weight_Nominal > robot Max_Load_Capacity, the matching degree is 0; if the item packaging type is incompatible with the robot Gripper_Type, the matching degree is 0; if the item contains liquid media and the robot is not equipped with a vibration sensor and microphone, the matching degree is low; if the item requires high-precision force perception and the robot force sensor is not accurate enough, the matching degree is reduced. The system traverses all combinations of items to be counted and all available robots, calculates the matching degree score, and forms a two-dimensional matching matrix M, where M ij represents the matching degree between the i-th item and the j-th robot. The system uses an allocation algorithm based on the matching matrix M (e.g., a cost-minimizing allocation algorithm) to select an optimal robot-item allocation scheme that maximizes the overall matching score or minimizes the total operation cost, subject to satisfying hard constraints (such as compatible payload and gripper types). This allocation scheme specifies which items or locations each robot is responsible for inventorying, and is referred to as the robot allocation result.

[0041] Based on each item ID, the system queries the item's material information or preset material type. The system stores a perception action rule library. This rule library is a lookup table whose key is the item's material type or a specific attribute combination (for example, a liquid container), and whose value is the recommended perception action type and detailed parameters for that item type. For example, the rule library entries are: Rule A: Material Type = Liquid; Recommended Action = Shake; Parameters = {'Amplitude': 5 degrees, 'Frequency': 2 Hz, 'Duration': 1 second}; Rule B: Material Type = Granular; Recommended Action = Vibrate; Parameters = {'Type': 'Sweep', 'Start Frequency': 50 Hz, 'End Frequency': 150 Hz, 'Duration': 1.5 seconds}; Rule C: Material Type = Solid Carton Packaging; Recommended Action = Squeeze; Parameters = {'Maximum Pressure': 10 N, 'Duration': 0.5 seconds, 'Deformation Limit': 5 ​​mm}. Each item in the robot's allocation results is iterated over. For each item, the system searches the perception action rule library based on its material type and extracts the corresponding perception action type and parameters. If there is no directly matching rule in the rule library for a certain item type, or if the responsible robot does not have the sensors required to perform the recommended action (for example, the rule recommends shaking, but the robot does not have a force / torque sensor), the system will fall back to the default perception action (for example, only basic grasping and lifting without additional micro-perturbations) or mark the item as unable to undergo in-depth perception. The system records each item ID, the corresponding perception action type, and detailed parameters into a list. This list is the perception action configuration table, which specifies the perceptual interaction operations that the robot should perform for each item to be counted.

[0042] The inventory routing plan specifies the order in which the robots must visit the locations. The robot assignment results determine which robots are responsible for accessing which locations. The sensing action configuration table specifies the sensing actions required to access specific locations and grasp items. The system constructs a structured data packet, which ultimately serves as the handling task instruction issued to the robot. This instruction packet contains: ① The unique task identifier (Task_ID): For example, this is generated based on a timestamp and sequence number, such as "INV_20231027_001." ② The executing robot identifier (Robot_ID): Specifies the robot that will receive and execute this task. ③ The operation type (Operation_Type): Specifies that this task is an "inventory operation." ④ The operation sequence (Operation_Sequence): This is an ordered list, with each element corresponding to a location access step in the routing plan. Each element contains: the target location identifier (Location_ID) and the item identifier (Item_ID), which includes batch information. Detailed sensing action instructions (Sensing_Action_Instruction): Contains the action type (such as 'shake', 'squeeze') and its specific execution parameters (such as amplitude, frequency, duration, pressure limit, etc.). These parameters are extracted from the sensing action configuration table. Target placement location (Target_Location): Usually the original storage location, but if the sensing analysis determines that the item is abnormal, it needs to be moved to the designated inspection area. Exception handling plan (Exception_Handling): For example, the fallback strategy in the event of a grasping failure or sensor failure. The above structured data packet is securely encrypted and transmitted to the onboard control system of the designated execution robot (determined by Robot_ID) through the warehouse network using a standard robot communication protocol (such as MQTT, ROS Action, or a custom TCP / IP protocol). This transmitted data packet is the handling task instruction.

[0043] Preferably, the inventory item grabbing process in step S2 includes:

[0044] Perform command parsing and sensor pre-activation on the handling task instructions to obtain the sensor configuration plan;

[0045] According to the sensor configuration plan and the storage location information in the handling task instruction, the items in the storage location are approached and positioned to obtain the grasping execution plan;

[0046] Initial contact and basic characteristics are acquired according to the crawl execution plan to obtain static contact characteristic data.

[0047] In this embodiment of the present invention, upon receiving a handling task instruction, the system first parses the instruction to identify the item's identity, location information, and the type and parameters of the required sensing action. Based on this information, the system then queries its internal sensor configuration rule library to determine which sensors (e.g., force / torque sensors, vibration sensors, and micro-microphones) need to be activated. It then sets appropriate operating modes for these sensors, including sampling frequency (e.g., 150Hz for force sensors, 500Hz for vibration sensors, and 16kHz for microphones), sensitivity (e.g., force sensors set to high sensitivity mode of ±0.01N), and data transmission format. This completes sensor pre-activation and generates a sensor configuration plan.

[0048] The robot then uses its navigation system to navigate to the target storage location based on the sensing requirements specified in the sensor configuration and the location information in the handling task instructions. Upon arrival, the robot uses its onboard vision system (e.g., a binocular camera or RGB-D camera) to capture images of the items in the storage location. The image processing module employs object recognition algorithms (e.g., the deep learning-based YOLOv7 or Mask R-CNN models) and pose estimation algorithms to locate the target object in the image and calculate its precise 3D position (x, y, z) and pose (roll, pitch, and yaw) relative to the robot's base coordinate system. The grasp planning module utilizes this precise pose information, along with the known dimensional model of the object and the geometric model of the robot's end-effector (gripper), to calculate the optimal grasp point and grasping pose. It also determines the pre-grasp position and approach velocity (e.g., 0.1 m / s) for the robot arm to reach the grasp point from its current position. These calculation results, including the precise position of the object, the coordinates of the optimal grasping point, the grasping posture, the approach trajectory, the preset initial contact force threshold (for example, 0.5N) and the grasping force parameters, together constitute the grasping execution plan.

[0049] The robot controls its arm movement according to the grasping execution plan, slowly approaching the object with its gripper. During this process, force / torque sensors monitor the forces acting on the gripper in real time. When the force detected by the sensors in a particular axis exceeds a preset initial contact force threshold (e.g., force Fz > 0.5N along the gripper's approach direction), the robot's motion control system determines that contact has occurred with the object, immediately halts the main approach motion, and switches to force-controlled or impedance-controlled mode, maintaining a gentle, stable contact force (e.g., keeping force Fz along the contact direction within the range of 2N ± 0.5N) for a short period of time (e.g., 1 second). During this stable contact period, all activated sensors (force / torque sensors, vibration sensors, and microphones) synchronously collect data at a high frequency (determined by the sensor configuration). The force / torque sensors record the reaction force curve of the object under stable contact force, reflecting the compliance and initial mechanical impedance of the object's surface. The vibration sensors and microphones collect transient vibration and acoustic signals generated during contact, as well as the baseline response of ambient noise transmitted through the object. These raw sensor data streams collected during the stable low-force contact phase, which reflect the static or quasi-static physical responses of the object, become static contact characteristic data after being timestamped and initially formatted.

[0050] Of particular importance are initial contact and basic feature acquisition, including:

[0051] Determine contact parameters based on the crawl execution plan;

[0052] Perform slow approach control according to contact parameters to obtain approach process records;

[0053] Perform initial contact detection based on the approach process record to obtain contact trigger data;

[0054] Perform constant force maintenance control according to the contact trigger data to obtain constant force control data;

[0055] Perform multi-point micro-deformation tests based on constant force control data to obtain static contact characteristic data;

[0056] In an embodiment of the present invention, a grasping execution plan is received. The plan is generated by the upper system based on the pose, type and capability of the object, and includes the preliminary pose of the robot end effector before it reaches the grasping point of the object, the approach trajectory vector from the preliminary pose to the grasping point (for example, a unit vector u, representing the negative direction along the Z axis of the robot end effector coordinate system), and the relevant parameters of the initial grasping operation to be performed. The system extracts the control parameters for the initial contact phase from the grasping execution plan. These parameters include: the initial contact detection threshold (for example, set to a force along the approach trajectory vector direction); >0.5N), where is the dot product F·u of the force vector F measured by the force sensor of the robot end effector and the approach trajectory vector u; the target constant contact force used in the stable contact phase (for example, set as the force along the approach trajectory vector = 2N); the time for which the stable contact force is maintained (for example, set to 1 second). These extracted and set values ​​together constitute the contact parameters. The robot first moves to a preliminary pose. Next, the robot control system executes a controlled linear motion command, slowly moving the end effector toward the object in the direction indicated by the approach trajectory vector u. This phase uses position control, but with very low speed limits (for example, the maximum movement speed is set to 0.05 m / s). While the robot performs the slow approach, its onboard multimodal sensors (force / torque sensor, vibration sensor, and micro-microphone) continuously collect data at a high frequency (for example, 1000 Hz for the force sensor, 500 Hz for the vibration sensor, and 16 kHz for the microphone). The robot control system simultaneously records the precise position (x, y, z coordinates) and attitude (roll, pitch, and yaw angles) of the end effector over time. All of this sensor data and robot status data, along with precise timestamps, are continuously recorded to form a record of the approach process. The system continuously calculates the component of the currently measured end effector force vector F in the direction of the approach trajectory vector u. =F·u. The system will calculate The value is compared with the initial contact detection threshold (e.g. 0.5N) set in the contact parameters. The value of jumps from below the threshold and remains above the threshold (for example, 10 consecutive sampling points >0.5N), the system determines that the robot end-effector has made effective contact with the object surface. At this point, the system immediately timestamps the current precise time and records the raw readings of all sensors (force / torque, vibration, and microphone) at the moment of the trigger, as well as the robot end-effector's pose information. This collection of information, marking the trigger time and the corresponding sensor / robot state, is the contact trigger data. The system then switches the robot end-effector's control mode from position control to force control (or other forms of impedance control or hybrid force / position control) along the approach trajectory vector u. The control objective is to apply and maintain a constant force along the u direction on the object, equal to the target constant contact force (e.g., 2N) set in the contact parameters. The control loop continuously receives real-time force data from the force / torque sensors and, based on the error between the target force and the actual measured force, adjusts the torque of the robot joints or the position of the end-effector to minimize the error, thereby achieving constant force output. During the set duration of constant force hold control (e.g., 1 second), all activated sensors (force / torque sensors, vibration sensors, and micro-microphones) continue to synchronously collect data at a high frequency. These raw sensor data streams acquired under constant contact force, along with a record of the robot end-effector's posture adjustments, are continuously recorded to form constant-force control data. The constant-force control data, which includes force / torque, vibration, and acoustic sensor data acquired under constant contact force, as well as robot posture records, is received. During this constant-force maintenance phase, the robot control system can also superimpose preset micro-perturbation commands. These micro-perturbations impose small, controlled displacements or force changes on the object without significantly changing the total contact force. For example, the system can instruct the robot to apply a small, sinusoidal force fluctuation of ±0.2N peak-to-peak and 0.1Hz in a direction perpendicular to the contact surface while maintaining an average contact force of 2N. Alternatively, the system can instruct the robot to attempt a small displacement (e.g., ±0.5mm) in one direction within the contact plane while maintaining a constant contact force. While executing these micro-perturbations, the force / torque sensor accurately records the object's dynamic response (i.e., the relationship between force and displacement or velocity) to these small force / displacement inputs at an extremely high frequency (e.g., 2000Hz). This additional high-frequency sensor data, reflecting the object's local mechanical properties, is integrated with all sensor data and robot status records collected during the entire constant force maintenance phase and, after timestamp calibration, forms the static contact characteristic data. This data reflects the object's fundamental physical properties, such as surface hardness, local stiffness, and damping, under static or quasi-static load conditions.

[0057] Preferably, the dynamic interaction and multimodal signal acquisition in step S2 includes:

[0058] Based on the static contact characteristic data, the grasping force is incrementally controlled to obtain the grasping process characteristic curve;

[0059] According to the characteristic curve of the grabbing process, the lifting gravity response analysis is carried out to obtain the lifting gravity response curve;

[0060] Evaluate the object's suspension stability based on the lifting gravity response curve to obtain suspension stability data;

[0061] Based on the suspension stability data, the perception action is executed and micro-perturbation enhancement is performed to obtain the perturbation action response data;

[0062] collecting sensor array data based on disturbance action response data;

[0063] Perform multimodal data time alignment on sensor array data to obtain time-synchronized data streams;

[0064] Perform dynamic response data integration on the disturbance action response data and the time synchronization data stream to obtain dynamic interaction response data;

[0065] Performing signal synchronization and noise reduction on the dynamic interactive response data to obtain a noise reduction signal data packet;

[0066] The noise reduction signal data packet is encapsulated and transmitted to obtain interactive perception data.

[0067] In an embodiment of the present invention, based on the target grasping force preset in the grasping execution plan (for example, 15N for a cardboard box), the robot control system initiates grasping force increment control. The control mode switches from the previous constant force maintenance to controlled force / position hybrid control or pure force control. The system instructs the robot's end effector (gripper) to gradually increase the gripping or supporting force on the object. The force increment process can adopt a linear increase mode (for example, linearly increasing the force from 2N to 15N within 0.5 seconds), a segmented increase mode, or even an adaptive force increment strategy based on the mechanical response of the object (derived from preliminary analysis of static contact characteristic data) to avoid excessive force. Throughout this force increment process, the robot's onboard high-precision force / torque sensor continuously collects triaxial force (Fx, Fy, Fz) and triaxial torque (Tx, Ty, Tz) data between the gripper and the object at a high frequency (for example, 1000Hz). These time-varying force / torque measurements accurately reflect the deformation, load-bearing capacity, and initial response of the internal structure of the item packaging under external forces, and are recorded to form a characteristic curve of the grasping process.

[0068] After confirming a secure grasp, the robot control system executes a vertical upward lift of the object. This lift utilizes a controlled velocity profile (for example, accelerating from 0.1 m / s² to 0.2 m / s² and then maintaining a constant velocity). The force / torque sensor continues to collect data at a high frequency throughout the entire process from the object's complete departure from the supporting surface (e.g., shelf) to its suspended state. The system analyzes the force / torque data during this phase, focusing specifically on the force Fz acting against gravity (typically in the negative Z-axis of the robot's base coordinate system). As the object accelerates upward, Fz will be greater than its actual weight. Once the object rises at a constant velocity and stabilizes in the air, its average value approaches the object's total weight (i.e., mass M × gravitational acceleration g, Fz ≈ Mg). Changes in force / torque during the lift, particularly the stable vertical force and fluctuations in horizontal force / torque, reflect the object's total mass, center of mass position, and whether the grasp is balanced. These force / torque time series data, reflecting the object's weight and grasp balance, constitute the lift gravity response curve.

[0069] After the object is completely suspended and stops vertically, the robot maintains its gripping posture, allowing the object to remain suspended. The system analyzes the force / torque sensor data during this suspended state. Evaluation indicators include: the fluctuation amplitude of the vertical force Fz (for example, calculating the standard deviation of Fz) ), which reflects whether the grasp is shaking or whether there is continuous small movement inside the object; the average value and fluctuation amplitude of the horizontal force components Fx, Fy and the torque components Tx, Ty, Tz (especially Tz around the vertical axis), which reflect whether the grasping point deviates from the center of mass of the object and whether the object rotates or tilts during the grasping process. If the average value of Fx, Fy or Tx, Ty is significantly different from zero, or their fluctuation amplitude ( , , , ) is large, indicating that the grasping is unstable or the object's center of mass deviates from the grasping center. If the fluctuation of Tz ( ) is large, indicating that there are rotating parts or liquid sloshing inside the item. The system calculates these statistics and compares them with pre-set stability thresholds. These quantitative stability assessment results (e.g., =0.05N, average Fx=0.1N, =0.02Nm), which constitutes the suspension stability data.

[0070] Based on the safety-constrained action plan generated in step S14, the system instructs the robot control system to perform a sensing action specific to the object (e.g., a slight shake or squeeze). During the sensing action, the robot's joint control system or end-effector force control system precisely drives the robot according to the safety-adjusted parameters specified in the plan (e.g., a shake amplitude of 5°, a frequency of 2Hz, and a squeeze pressure of 10N). To enhance sensor signals, the system can superimpose micro-perturbations on top of the main sensing action. For example, while squeezing, a sequence of small, high-frequency (e.g., 100Hz) force pulses is applied to the squeeze surface, or while shaking, a series of small vibrations along different axes are superimposed. During the entire set duration (e.g., 1.5 seconds) of the sensing action and superimposed micro-perturbations, all activated multimodal sensors (force / torque sensors, vibration sensors, and micro-microphones) continuously collect raw data at their configured high frequencies (e.g., 1000Hz for force sensors, 500Hz for vibration sensors, and 16kHz for microphones). These sensor data streams record the physical response of the object to the dynamic interactions and micro-perturbations actively applied by the robot, constituting the perturbation action response data.

[0071] When executing sensing actions and performing micro-disturbance enhancement based on airborne stability data, the robot's onboard sensor array, including at least force / torque sensors, vibration sensors, and micro-microphones, synchronously and frequently collects the object's response signals to these dynamic inputs. These sensors are distributed at different locations on the robot's gripper arm or end effector, forming a sensor array. Each sensor operates independently, collecting a data stream of its specific modality and generating data packets with local timestamps. The collection of these raw data streams collected in parallel from the sensor array is referred to as the sensor array data.

[0072] Because different sensors use independent internal clocks or experience varying data transmission delays, their raw timestamps may not be completely consistent. For subsequent multimodal data fusion analysis, these data streams must be precisely aligned to a common global timeline. The system utilizes a high-precision time synchronization mechanism. One approach is to trigger data acquisition from all sensors using a synchronization pulse from the robot's main control system to ensure simultaneous sampling. Another approach utilizes the hardware timestamp contained in each sensor data packet and performs post-processing alignment in the main control system or data processing unit. Post-processing alignment employs an interpolation algorithm to map the local timestamp of each data point onto a unified global timeline. Resampling is then performed to generate multimodal data vectors aligned at each global time point. Alignment accuracy typically requires millisecond or even sub-millisecond accuracy to preserve the transient characteristics of the signals. The processed data, in which data points from different sensors precisely correspond to each other on the timeline, forms a time-synchronized data stream.

[0073] The system integrates other relevant data generated during the dynamic phases of the inventory operation (grasping, lifting, suspending, and sensing), including grasping process characteristic curves, lifting gravity response curves, suspending stability data, and the precise motion command parameters and actual execution trajectory records when the robot performs sensing actions. The system organizes all this data, including raw and pre-processed (e.g., time-aligned) sensor data, robot state data, and preliminary features or evaluation results calculated based on this data, into a unified, structured data package. This data package contains a complete record of all dynamic interactions between the robot and the object, and the resulting multimodal sensor responses, from the time the object is grasped to the completion of the sensing action. This structured data package, which aggregates information related to all dynamic phases, is the dynamic interaction response data.

[0074] Receive dynamic interaction response data. Although time alignment was performed in the previous step, this step allows for more refined signal synchronization verification, ensuring that all modal data remain aligned on a microscopic timescale. More importantly, this step performs signal noise reduction. For force / torque and vibration signals, digital filters (e.g., Butterworth filters) are applied to remove high-frequency random noise and specific frequency interference introduced by the robot's own motion or environmental vibrations (e.g., notch filters are used to remove known resonant frequencies of the robot's motors or joints). For acoustic signals, techniques such as spectral subtraction or non-negative matrix factorization are applied, leveraging the ambient baseline noise data collected in step S2.2 (approach and localization preparation) to separate and suppress background noise from the acoustic signals generated by object interaction. Noise reduction algorithm parameters are optimized based on sensor type and environmental conditions. Noise reduction aims to improve the signal-to-noise ratio and highlight the inherent response characteristics of the object. After fine synchronization and noise reduction, the multimodal sensor data stream is encapsulated into a noise-reduced signal data packet.

[0075] The noise reduction signal data packet is finally packaged. This package structure includes the item's unique identifier (Item_ID), the robot identifier performing the inventory operation (Robot_ID), the start timestamp of the interactive perception process, the type and parameters of the perception action performed, and a standardized format block containing the noise-reduced sensor time series data, including force / torque, vibration, and acoustics. Sensor data can be compressed using a compression algorithm (e.g., lossless compression algorithms such as LZ4 or lossy compression algorithms such as Opus for audio, provided that key features are preserved) to reduce the data volume. The packaged data packet is named interactive perception data and transmitted via the warehouse's internal network (e.g., a secure communication channel based on the TCP / IP protocol) to a central processing system or a dedicated data analysis server for subsequent intelligent material perception analysis (step S3). Error detection and retransmission mechanisms are implemented during transmission to ensure data integrity.

[0076] Preferably, the sensing signal separation in step S3 includes:

[0077] Perform signal separation and preprocessing on interactive perception data to obtain classification signal feature set;

[0078] Extract mechanical characteristic parameters from the classification signal feature set to obtain a mechanical parameter feature table;

[0079] An acoustic feature map is generated for the classification signal feature set to obtain an acoustic feature map.

[0080] In an embodiment of the present invention, the data streams of different sensor modes are separated and extracted according to the predefined format and identifier in the data packet to form three independent original signal sets: a force / torque signal set, a vibration signal set, and an acoustic signal set. Then, each signal set is subjected to modality-specific preprocessing. For the force / torque signal set, a 4th-order Butterworth low-pass filter is applied with a cutoff frequency set to 50 Hz to filter out high-frequency noise caused by high-speed sampling. For the vibration signal set, a 4th-order Butterworth bandpass filter is applied with a passband range set to 20 Hz to 200 Hz to focus on the characteristic frequency bands generated by the vibration of the object structure and the movement of the contents. For the acoustic signal set, the short-time Fourier transform (STFT) method is adopted, using the Hanning window function, the window length is set to 1024 sampling points, and the overlap rate is 50% to generate a time-frequency representation of the acoustic signal, i.e., a spectrogram. All filtered and transformed signal data is then amplitude normalized, for example, using the Z-score method to convert the signal values ​​to a distribution with a mean of 0 and a standard deviation of 1. This ensures comparability between different modalities and objects. All of this separated and modality-specific preprocessed data is organized into a classification signal feature set, which includes a force / torque signal array, a vibration signal array, and an acoustic spectrogram matrix.

[0081] Extract parameters reflecting the mechanical properties of the object from the force / torque signal array. For example, the average value of the vertical force component (e.g., Fz) during the uniform lifting phase is used to estimate the object's equivalent mass (mass ≈ average Fz / gravitational acceleration g). The force versus gripper displacement curve during the incremental grasping force is analyzed, and its slope is calculated. This slope reflects the local stiffness of the object packaging. The slope of the vertical force (Fz) versus robot angular acceleration (α) during the shaking motion is analyzed. This slope reflects the equivalent moment of inertia of the movable medium (e.g., liquid, particles) within the object and can be used as an indicator of internal filling volume. The squeezing force versus squeezing displacement curve during the squeezing motion is analyzed, and its nonlinear characteristics or elastic recovery ratio are calculated to reflect the elasticity and compressive resistance of the packaging. From the vibration signal array, after the object is struck or shaken, the decay rate of the vibration amplitude is analyzed to calculate the energy attenuation coefficient of the vibration signal. This coefficient reflects the damping characteristics of the object's internal structure. All these calculations yield quantitative values ​​such as estimated mass (in kg), package stiffness (in N / mm), equivalent moment of inertia (in kg· ), elastic recovery ratio (unitless quantity), and energy attenuation coefficient (unit 1 / s) are organized into a structured list or table, where each row corresponds to a set of mechanical characteristic parameters of an item, forming a mechanical parameter characteristic table.

[0082] Extract acoustic features reflecting the internal state of the object from the acoustic spectrogram matrix. For example, calculate the total energy in a specific low-frequency band (e.g., 5Hz-15Hz), which is related to the intensity of liquid sloshing. Calculate the total energy in another specific frequency band (e.g., 50Hz-100Hz), which is related to particle collision or sliding. Analyze the temporal changes of the spectrogram's spectral centroid (indicating the "brightness" or center of the frequency distribution of the spectrum) to reflect the changing characteristics of the sound. Calculate the spectrogram's spectral flux (a measure of the change in spectral shape between consecutive frames) to reflect the dynamic nature of the sound. Detect the presence of significant resonance peaks in the vibration signal's spectrum and identify their dominant frequencies and bandwidths, which reflect the inherent vibration modes of the object as a whole or its internal structure. Analyze the amplitude envelope of the acoustic signal to calculate the sound's onset, duration, and decay time. For example, the sound of liquid sloshing typically lasts longer and decays slowly, while the sound of a solid impact decays rapidly. The system integrates these extracted acoustic and vibration features, such as the energy ratio (unitless) in a specific frequency band, spectral centroid (unit: Hz), spectral flux (unitless), resonant frequency (unit: Hz), resonant peak bandwidth (unit: Hz), and sound duration (unit: seconds), into a multidimensional feature vector or matrix. This vector or matrix quantitatively describes the acoustic and vibration response characteristics of an object and can be considered the object's acoustic signature map, which is used for subsequent object identification and internal state determination.

[0083] Preferably, the acoustic feature map generation includes:

[0084] Perform acoustic and vibration time-frequency conversion on the classification signal feature set to obtain the time-frequency energy matrix;

[0085] Divide the material characteristic frequency bands based on the time-frequency energy matrix to obtain the frequency band statistical characteristic table;

[0086] Perform spectral centroid and morphology calculations on the time-frequency energy matrix to obtain a spectral morphology parameter sequence;

[0087] Performing time-varying analysis of the acoustic energy envelope on the classification signal feature set to obtain an envelope dynamic parameter table;

[0088] Perform acoustic fingerprint matching calculation on the frequency band statistical feature table, spectrum morphology parameter sequence and envelope dynamic parameter table to obtain the material acoustic feature matching table;

[0089] Extract filling rate acoustic markers based on the material acoustic feature matching table and envelope dynamic parameter table to obtain a filling state acoustic index table;

[0090] Perform abnormal resonance and acoustic deviation detection on the time-frequency energy matrix to obtain an abnormal acoustic feature table;

[0091] Generate an acoustic feature map based on the abnormal acoustic feature table and the filling status acoustic index table.

[0092] In this embodiment of the present invention, a comprehensive approach of time-frequency analysis, spectral feature calculation, time-varying envelope analysis, and pattern matching techniques is employed to extract and quantify acoustic features reflecting the internal structure and state of an object from these signals. First, a time-frequency conversion is performed on the sound and vibration signals to generate a matrix describing the distribution of signal energy in time and frequency. Next, based on this matrix, the system identifies and statistically analyzes the energy distribution within frequency response bands specific to different material types. Simultaneously, statistical moments (such as centroid and bandwidth) and morphological parameters of the spectrum within each time frame are calculated to describe the shape of the spectrum. Furthermore, the time-varying envelope of the sound signal's total energy is analyzed to extract its dynamic parameters. These calculated quantitative features are then compared with a pre-defined database of acoustic fingerprint templates to identify the object's acoustic type. Furthermore, acoustic indicators related to fill rate are further analyzed to detect any abnormal resonances or acoustic deviations that are inconsistent with expectations. Finally, these multi-dimensional, quantitative acoustic analysis results are integrated to generate an acoustic signature map for the object.

[0093] For acoustic signals, the system uses the short-time Fourier transform (STFT) method for time-frequency analysis. The analysis window type is set to a Hamming window, with a window length of 1024 samples and a frame shift (the interval between the starting points of adjacent windows) of 512 samples (i.e., 50% overlap). A fast Fourier transform (FFT) is performed on the signal segment within each time window, its frequency spectrum is calculated, and the square of the amplitude is taken to obtain the energy spectrum. The energy spectra of consecutive time windows are arranged in chronological order to form a two-dimensional matrix E_acoustic, whose rows represent time frames and columns represent frequency bins. The matrix elements E_acoustic[t,f] represent the energy at frequency f corresponding to time frame t. For vibration signals, the STFT method is also used, with a window length of 256 samples and a frame shift of 128 samples. An FFT is performed on the vibration signal (for example, the vector norm of the three-axis acceleration) in each time window, and the energy spectrum is calculated to obtain the vibration signal's time-frequency energy matrix E_vibration. The system can combine E_acoustic and E_vibration into an overall time-frequency energy matrix containing the energy distribution of different modes, or process them as two independent matrices, but both marked with the same global timestamp.

[0094] The system stores a library of characteristic frequency band definitions for different material types. For example, the frequency band library defines: liquid sloshing frequency band L (5Hz-15Hz), particle sliding / collision frequency band G (50Hz-100Hz), hard structure resonance frequency band (150Hz-250Hz), packaging material resonance frequency band (80Hz-120Hz). The system traverses each time frame of the time-frequency energy matrix. For each time frame, the system calculates the total energy or average energy in each predefined characteristic frequency band. For example, for time frame t and liquid sloshing frequency band L, calculate ∑E[t,f], where f belongs to the frequency range corresponding to frequency band L. In addition to the total energy, statistics such as the energy peak and the standard deviation of the energy in each frequency band can also be calculated. These statistics reflect the activity level of the object in different characteristic frequency ranges at a specific point in time. The system organizes these statistics calculated by time frame and frequency band into a table or matrix, such as a three-dimensional matrix [time frame × frequency band × statistic type], to form a frequency band statistical feature table.

[0095] Traverse each time frame of the time-frequency energy matrix. For each spectrum corresponding to time frame t (for example, acoustic energy spectrum E_acoustic[t,:]), the system calculates the statistics describing its shape. Calculate the spectral centroid (SpectralCentroid) using the formula =(∑ ×E[t, ]) / (∑E[t, ]),in Is the frequency corresponding to the frequency bin. Calculate the spectral spread to reflect the degree of energy dispersion on the frequency axis. Calculate the spectral skewness and kurtosis to reflect the symmetry and peak state of the energy distribution. Calculate the spectral flatness to reflect whether the spectrum is biased towards noise (flat) or towards tone (sharp). These calculations obtain a set of values ​​per time frame to form a parameter vector that changes over time. Arrange the parameter vectors of all time frames in order to form a matrix [time frame × morphological parameter type], which is the spectral morphological parameter sequence. For example, the sequence contains 、 、 The trajectory of the parameters changing with time t.

[0096] The energy envelope of an acoustic signal is calculated. One method involves rectifying the signal (taking its absolute value) and then smoothing it through a low-pass filter (e.g., with a cutoff frequency of 10Hz). This yields a curve that reflects the temporal variation of the sound's loudness. The system then analyzes the dynamic characteristics of this envelope. Extracted parameters include: total duration (the interval from the start to the end of the sound, e.g., the point at which the energy envelope drops below 10% of its peak value); attack time (the time from the start of the sound to the peak of the envelope); decay time (the time required for the envelope's peak value to fall to a certain percentage (e.g., 50% or 10% of the peak value)); peak amplitude of the envelope; RMS (root mean square) value of the envelope; and characteristics of the envelope's shape (e.g., whether it contains multiple distinct peaks and the intervals between peaks). These calculated quantitative values, such as total duration (in seconds), attack time (in seconds), decay time (in seconds), and peak amplitude (unitless or relative), are organized into a table to form the envelope dynamic parameter table.

[0097] These features from different analysis dimensions are integrated to form a high-dimensional object acoustic feature vector. For example, the time-frame average results from the frequency band statistical feature table, the average of the spectral morphology parameter sequence or a snapshot at a specific time point, and all parameters from the envelope dynamic parameter table can be concatenated to form a comprehensive feature vector F_acoustic. The system internally stores a database of known acoustic fingerprints for object / material types. This database contains typical acoustic feature vectors or feature distribution models (e.g., Gaussian mixture models) corresponding to various known object types. The system uses a pattern matching algorithm to compare the current object's comprehensive feature vector F_acoustic with each known acoustic fingerprint in the database. This matching can be performed using various methods, such as calculating the Euclidean distance or cosine similarity between F_acoustic and each typical feature vector in the database. If the database stores feature distribution models, the likelihood of F_acoustic under each model is calculated. The system sorts the items based on the matching degree or likelihood, generating a list of known item types that are most similar to the current item's acoustic features and their corresponding matching scores (e.g., a similarity score of 0-1). The type with the highest score is considered the preliminary recognition result. The system records the recognition results, including the identified item type identification, the highest matching score, the second highest matching score and other information, to form a material acoustic feature matching table.

[0098] The system checks the item type identified in the material acoustic signature matching table to determine whether it is a container-type item for which fill rate estimation is required (e.g., liquid container, granular container). If not, this step is skipped. If the item is a container-type item, the system extracts acoustic metrics highly correlated with the fill rate from the envelope dynamic parameter table and / or frequency band statistical feature table based on the specific item type matched. For example, for liquid containers, the system extracts the peak energy, energy decay time, and envelope duration in the sloshing frequency band (5Hz-15Hz). For granular containers, the system extracts the energy in the particle collision frequency band (50Hz-100Hz), the damping coefficient of the vibration signal, and the rapid decay characteristics of the sound envelope. The system stores internal mapping models between acoustic metrics and fill rates for different container types and contents. These models can be regression models trained based on experimental data (e.g., a linear regression model that calculates Fill_Rate = a × Peak_Energy + b × Decay_Time + c) or pre-calibrated lookup tables. The system inputs the extracted acoustic indices into the corresponding models to calculate quantitative indicators reflecting the filling state. These indicators are either direct estimated filling fraction percentages or intermediate quantities that are strongly correlated with the filling fraction (e.g., equivalent sloshing mass, equivalent particle density). The system records these calculated filling-state-related acoustic indices and / or preliminary filling fraction estimates to form a filling-state acoustic indicator table.

[0099] The system obtains the item type identified in the material acoustic signature matching table and queries the "normal" acoustic signature template or statistical distribution for that type of item (e.g., the energy range of key frequency bands under normal conditions, expected resonant frequencies). The system compares the current item's time-frequency energy matrix with the normal template to detect any anomalies. Detection methods include: ① Abnormal resonance detection: This system searches the entire frequency spectrum for significant energy peaks. These peaks correspond to frequencies that are inconsistent with the natural frequencies of a normal item, or whether the Q factor (the sharpness of the peak, reflecting damping) is abnormally high or low. For example, if a sharp resonant peak in a high-frequency band of an item that should be solid is present, it indicates an internal cavity or loose components. ② Acoustic deviation detection: This system calculates a metric (e.g., using KL divergence or spectral distance) between the current item's time-frequency energy distribution and the normal template. If the difference exceeds a preset threshold, the item is flagged as an acoustic deviation. This system detects sudden, brief high-energy events, such as abnormal noise generated by internal component breakage or foreign object collision. The system quantifies these anomalies, for example, recording the frequency, amplitude, and bandwidth of abnormal resonances, the time and intensity of abnormal transient noise, and calculating an overall acoustic deviation score. This quantified anomaly information is organized into a table or list to form an abnormal acoustic signature table.

[0100] All key analysis results regarding the item's acoustic properties are integrated. These results include: the item type (and its confidence level) identified in the Material Acoustic Signature Matching Table, the fill quantity or related indicators reflected in the Fill Status Acoustic Indicator Table, and any anomalies detected in the Abnormal Acoustic Signature Table (such as abnormal resonance, abnormal noise, and overall deviation). The system organizes this information into a structured data object, which is the item's acoustic signature map. The acoustic signature map is a multi-dimensional representation that not only includes the item's type identification results, but also includes in-depth analysis of its "internal sounds" and a quantitative assessment of its fill status. By comprehensively processing the data from the Abnormal Acoustic Signature Table and the Fill Status Acoustic Indicator Table, a more complete acoustic signature description is generated, ensuring that the map contains both information about the item's abnormal state and accurate fill quantity estimates. For example, an acoustic feature map can include the following fields: item ID, identified acoustic type (e.g., "liquid container"), type confidence (e.g., 0.92), filling status indicators (e.g., "estimated filling rate: 75%," "related indicators: high shaking energy, slow decay"), internal state anomaly flags (e.g., "abnormality present"), and anomaly details (e.g., "high-frequency abnormal resonance detected @ 180Hz, abnormally high Q factor, suspected internal structure abnormality"). This map, which integrates anomaly detection results and filling status assessment, provides a comprehensive and detailed basis for subsequent inventory information updates and status assessments based on acoustic perception.

[0101] Preferably, the quantity / filling amount estimation and perception report generation in step S3 includes:

[0102] Extracting a filling-related parameter set from an item identified as a container in an item type determination result;

[0103] Matching the populated relevant parameter set with the pre-established physical model parameter library to obtain the estimation model selection result;

[0104] Calculate the liquid filling volume based on the estimation model selection result to obtain the calculated value of the liquid filling rate;

[0105] The particle number is estimated based on the estimation model selection result to obtain the particle filling rate calculation value;

[0106] Based on the estimation model selection results, solid object count estimation is performed to obtain an estimated value of the number of items;

[0107] Estimation reliability scores are performed on the liquid filling rate calculation value, the particle filling rate calculation value, and the item quantity estimation value to obtain an estimation reliability score;

[0108] Calibrate estimated reliability scores with historical data and generate fill volume estimates;

[0109] A comprehensive report is generated based on the item type determination results, internal state evaluation table, and filling quantity estimation results to obtain a material perception feature report.

[0110] In this embodiment of the present invention, the analysis module receives the item type determination results, the internal state assessment table, and various feature data generated in the previous steps (S31-S33) (mechanical parameter feature table, acoustic feature map, filling state acoustic index table, etc.). The system first identifies items requiring quantity or fill level estimation from the item type determination results, namely, those items identified as containers (e.g., liquid containers, granular containers) or containing countable solid items. For these items, the system extracts a series of quantitative parameters related to the item's contents, such as mass, volume, density, sloshing characteristics, vibration damping, and acoustic response, from the mechanical parameter feature table and the filling state acoustic index table to form a filling-related parameter set. Next, based on the specific item type and the characteristics of the filling-related parameter set, the system searches and matches the most suitable model or algorithm for estimation within a pre-established physical model parameter library, obtaining an estimation model selection result. Based on the selected model, the system performs corresponding estimation calculations for liquid, granular, or solid items, respectively, obtaining preliminary calculated values ​​(e.g., liquid filling rate, granular filling rate, and item quantity estimate). To assess the reliability of these estimates, the system calculates an estimate reliability score, taking into account factors such as sensor data quality, item type identification confidence, and the consistency of information provided by different perception modalities (mechanical, acoustic). The system then calibrates the estimate reliability score and / or preliminary estimate using historical inventory data and corresponding perception estimation results, generating a more accurate fill quantity estimate that includes the final estimate value and the estimated error range. Finally, the system integrates the item type determination results, internal condition assessment table, and fill quantity estimation results into a structured document to generate a complete material perception feature report.

[0111] The system checks whether the identified item type falls into the pre-defined "estimated fill" or "estimated quantity" categories. If the item type is in the list, the system extracts key parameters related to the contents' state and quantity from the item's corresponding mechanical parameter feature table (including estimated mass M_est, estimated moment of inertia I_est, packaging stiffness K_package, vibration damping coefficient ζ) and filling state acoustic indicator table (including shake frequency energy E_shake, granular frequency energy E_granular, acoustic energy decay time τ_decay, specific resonance frequency f_resonance, etc.). For example, for "liquid container_model A," M_est, I_est, E_shake, and τ_decay are extracted; for "granular bag_model B," M_est, ζ, and E_granular are extracted; and for "small box_containing 10 solid parts," M_est and f_resonance (reflecting internal looseness or count) are extracted. These extracted quantitative values ​​are organized into a filling-related parameter set for each item requiring estimation.

[0112] Based on the item's identification type, the system searches for a matching model in the physical model parameter library. For example, if the item is identified as "Liquid Container_Model A", the system searches for a model list that applies to "Liquid Container_Model A" (e.g. , , ). The system further checks which input parameters required by the model are included in the relevant parameter set, and evaluates the parameter quality (for example, the signal-to-noise ratio of the sensor signal). The system selects the model with the highest priority, complete required parameters and qualified quality as the estimation model for the current item. If multiple models are matched, the system can select the best one according to preset rules (for example, select a model based on more perceptual modality data), or mark it as a model that can be used for redundant estimation using multiple models. The selected model identifier and the list of parameters required by it constitute the estimation model selection result.

[0113] All the input parameters required by the model (e.g., M_est, I_est, E_shake, τ_decay) are extracted from the filling-related parameter set. The system queries the item master data to obtain the known properties of the liquid container, such as the empty container mass M_empty, the total volume V_total, the container height H_container, and the liquid density ρ_liquid. The system applies the liquid filling amount calculation formula or algorithm specified in the estimation model selection result. For example, if the model is selected, and the formula is: Estimated liquid mass M_liquid_est= (M_est,M_empty) and estimated liquid moment of inertia I_liquid_est= (I_est,I_empty), where , is the conversion function, I_empty is the moment of inertia of the empty container. Then, based on the liquid sloshing physical model, the filling height h_est is estimated. (I_liquid_est, M_liquid_est, H_container). The final calculation of liquid filling rate: Fill_Rate_Liquid=(h_est / H_container)×100%. If the model Using acoustic parameters, the formula is: Fill_Rate_Liquid= (E_shake, τ_decay, V_total). The system performs a calculation and obtains a percentage value, which is the calculated value of the liquid filling rate.

[0114] All the input parameters required for the model (e.g., M_est, ζ, E_granular) are extracted from the filling-related parameter set. The system queries the item master data to obtain the known properties of the granular container, such as the empty container mass M_empty, the total volume V_total, and the bulk density range ρ_bulk_range of the granular material. The system applies the granular number / filling amount calculation formula or algorithm specified in the estimation model selection result. For example, if the model If selected, the formula is: Estimate the granular mass M_granular_est = M_est - M_empty. Estimate the granular volume V_granular_est = M_granular_est / ρ_bulk_average, where ρ_bulk_average is the typical bulk density of the granular material. Finally, calculate the granular filling rate: Fill_Rate_Granular = (V_granular_est / V_total) × 100%. If the model Using acoustic and vibration parameters, the formula is: Fill_Rate_Granular= The system performs the calculation and obtains a percentage value (filling rate) or a number value (in the case of discrete particle counting), which is the calculated particle filling rate.

[0115] All input parameters required for the model (e.g., M_est, f_resonance, internal structure characteristic flags) are extracted from the populated parameter set. The system queries the item master data to obtain the known properties of the solid item, such as the mass of a single standard item, M_single. The system applies the solid item count calculation formula or algorithm specified in the estimation model selection result. For example, if the model The mass-based estimate is selected, and the formula is: the number of items estimated N_est=round((M_est-M_empty_package) / M_single), where M_empty_package is the empty mass of the package. Based on internal resonance or shock count (when shaking causes internal items to collide with each other to produce detectable shock) is selected, the formula is: N_est= (f_resonance, Number_of_Impacts), where Number_of_Impacts is the number of impact events detected from the acoustic / vibration data. The system performs a calculation and obtains a single value, which is the estimated number of items.

[0116] Calculate the reliability score of each estimation result. The scoring basis includes: ① The signal-to-noise ratio of the input sensor data: the higher the signal-to-noise ratio, the higher the reliability score. ② Confidence in item type identification: the higher the confidence in type determination, the higher the reliability score for estimation using the corresponding model of that type. ③ Consistency of multimodal data: for example, if the estimation quality is consistent with the indication direction of the acoustic analysis results (such as shaking energy) on the filling rate, the reliability score increases; if not, the reliability score decreases. ④ The inherent accuracy of the model used: the historical verification accuracy of each model is recorded in the physical model parameter library; the reliability score is higher when a model with higher accuracy is used. ⑤ The significance of the original signal features: for example, if the characteristic signal generated by the shaking of the liquid is very weak, even if it is identified as a liquid, its filling rate estimation reliability score is low. The system combines these factors and uses a preset scoring function Reliability_Score= (Signal_SNR,Type_Confidence,Modality_Consistency,Model_Precision,Feature_Salience) calculates a value between 0 and 1, where 1 represents the highest reliability. The system generates a corresponding estimation reliability score for each estimation performed (liquid filling fraction, particle filling fraction, solid number).

[0117] The system accesses a historical inventory database, which stores data from past manual counts or high-precision measurements, as well as estimates and various feature data obtained through sensing methods at the time. The system uses this historical data to build a calibration model. The calibration model can be a simple linear regression model (for example, final estimate = a × preliminary estimate + b) or a more complex machine learning model. It takes the preliminary estimate, estimated reliability score, item type, and some of the original feature data as input and outputs a corrected final estimate and an estimated error range. The goal of model training is to minimize the difference between the corrected estimate and the historical true value. Based on the current item type and estimated reliability score, the system applies the corresponding historical data calibration model to correct the preliminary estimate. For example, if historical data shows that for a certain liquid type, when the reliability score is 0.7, the estimate is typically 5% lower, the system will increase the preliminary calculated fill rate by 5%. The calibration model also outputs an estimated error range, such as ±5%. The final fill level estimate includes the calibrated estimate value (e.g., "Liquid fill rate: 78%" or "Number of items: 9"), the estimated error range (e.g., "±5%" or "±1 item"), and the estimate reliability score.

[0118] These analysis results are integrated into a structured material perception feature report document. The report structure includes, but is not limited to: report generation time, item unique identifier (Item_ID), item location (Location_ID), perception-based item type determination results (including the identified type and confidence level), perception-based internal state assessment results (including status tags such as "normal" and "abnormal," specific abnormality types such as "abnormal liquid sloshing," "suspected internal looseness," and abnormality severity scores), and perception-based quantity / fill volume estimation results (including estimated values, estimated error ranges, and estimated reliability scores). The report can also include metadata about the raw perception data (such as collection time and the type of perception action performed) as well as recommended actions generated by the system based on these results (for example, "inventory normal," "manual quantity verification required," or "move to quality inspection area"). The system outputs this structured document containing comprehensive perception analysis results as the material perception feature report.

[0119] Preferably, the synchronous updating of inventory information in step S4 includes:

[0120] Receive and verify the material perception characteristic report and obtain verification result records;

[0121] Compare the inventory data with the material perception feature report based on the verification result records to obtain a data difference analysis table;

[0122] Update inventory records based on the data difference analysis table to obtain updated data packets;

[0123] Evaluate the update permissions and priorities of the update data packages to obtain a hierarchical update plan;

[0124] The database transaction is executed according to the hierarchical update scheme to obtain the database update result.

[0125] In an embodiment of the present invention, the format, integrity, and content of the report are verified to ensure its validity, and a record of the verification results is obtained. Based on the verified report, the system compares it with the records in the current inventory database to identify whether there are any differences in information such as item type, quantity / filling amount, and internal status, and generates a data difference analysis table. Based on these differences and preset update rules, the system prepares instructions for modifying the inventory database to form an update data packet. The system then evaluates the authority requirements and business urgency of each modification in the update data packet, determines which updates can be executed automatically and which require manual approval, assigns priorities, and generates a hierarchical update plan. Finally, according to the hierarchical update plan, the system executes the approved update operations through the database transaction processing mechanism to ensure data consistency, obtain the database update results, and record the update details.

[0126] The material sensing feature report is transmitted in a structured format (for example, a JSON object) and includes fields such as item ID (Item_ID), location ID (Location_ID), sensing timestamp (Sensing_Timestamp), sensed item type (Sensed_Item_Type), internal state assessment result (Sensed_Internal_State, for example, including a state code and description), quantity / filling quantity estimation value (Sensed_Quantity_Value) and its unit, estimation error range (Sensed_Quantity_Error), and estimation reliability score (Sensed_Reliability_Score). The Data Integration Service module performs a series of validation rules on received reports: ① Format Validation: Checks whether the data structure conforms to the predefined JSON Schema; ② Required Field Check: Verifies that all mandatory fields (such as Item_ID, Location_ID, and Sensing_Timestamp) exist and are not null; ③ Identifier Validation: Query the item master data table to confirm whether the Item_ID is valid, and query the location table to confirm whether the Location_ID is valid; ④ Timestamp Validity: Verifies that the Sensing_Timestamp is within a reasonable window of the current time (for example, no more than 5 minutes in the past); ⑤ Data Range Check: Verifies that numeric fields (such as Sensed_Quantity_Value and Sensed_Reliability_Score) are within their valid range (for example, reliability score between 0 and 1, fill rate between 0% and 100%). Any validation failure is recorded, and a validation result record is generated containing the failure reason. Reports that successfully validate are marked as ready for further processing, and a validation result record is generated containing a success mark.

[0127] If the validation result record indicates that the report validation is successful, the system extracts the Item_ID and Location_ID from the material sensing feature report. The system retrieves the current inventory record for the item at the specified location from the Inventory_Main table in the warehouse inventory management system by executing a database query statement (for example, SELECT Current_Qty,State,Item_Type, Last_Inventory_Time FROM Inventory_Main WHERE Item_ID=? AND Location_ID=?). The system compares the sensed data in the report with the existing data in the database item by item: ① Item type comparison: Compares Sensed_Item_Type with Inventory_Main.Item_Type for consistency; ② Quantity / Filling amount comparison: Calculates the absolute difference between Sensed_Quantity_Value and Inventory_Main.Current_Qty (|Sensed_Quantity_Value - Inventory_Main.Current_Qty|) and the relative difference percentage (|Sensed_Quantity_Value - Inventory_Main.Current_Qty| / Inventory_Main.Current_Qty) × 100% (if Current_Qty > 0); ③ Internal state comparison: Compares the status code or description of Sensed_Internal_State with the Inventory_Main.State field; specifically marks situations where the status changes from "normal" to "abnormal"; ④ Count time recording: Records the Sensing_Timestamp in the report as the new candidate value for the last count time. All these comparison results, including difference values, difference percentages, status change marks, etc., are recorded in a structured manner to form a data difference analysis table for the item.

[0128] The specific database update instructions generated are determined by a pre-defined set of update rules based on the type and size of the discrepancy. For example, the rule set includes: Rule A: If the item type comparison result is inconsistent, an update instruction is generated for the Type field, with the new value being Sensed_Item_Type; Rule B: If the relative discrepancy percentage between the quantity and fill amount comparison is greater than a preset threshold (e.g., 5%), an update instruction is generated for the Current_Qty field, with the new value being Sensed_Quantity_Value; Rule C: If the internal status comparison shows a change from "normal" to "abnormal," an update instruction is generated for the State field, with the new value being Sensed_Internal_State, and an insert instruction is generated into the Inventory_State_History table, recording a detailed description of the abnormality and the sensing timestamp; Rule D: Regardless of whether there is a discrepancy, an update instruction is generated for the Last_Inventory_Time field, with the new value being Sensing_Timestamp. The system traverses the data discrepancy analysis table, applies the corresponding update rules, and constructs one or more database operation instructions (e.g., SQL UPDATE or INSERT statement objects). These commands, along with the Item_ID and Location_ID they target, and the source information of the update (such as the report ID), are packaged into an update packet.

[0129] Each database operation instruction in the update data package is evaluated for permissions and priority based on a preset update policy matrix. The update policy matrix defines how different types of update operations (e.g., quantity changes, status changes, and change types) are handled and prioritized under different conditions (e.g., quantity discrepancy size, status anomaly severity, item criticality, etc.). For example, the policy matrix stipulates: ① If a quantity adjustment is less than 5% and the perceived reliability score is greater than 0.8, it is marked as "auto-execute" with a "low" priority; ② If a quantity adjustment is greater than 20% or the item type does not match, it is marked as "requires manual review" with a "medium" priority; ③ If the internal status changes to a "serious anomaly" (e.g., a damaged liquid container), it is marked as "auto-execute" with a "high" priority, triggering subsequent alerting processes; ④ If the internal status changes to a "minor anomaly" (e.g., suspected internal looseness), it is marked as "auto-execute" with a "medium" priority, and the item is marked for manual inspection upon its next shipment. The system iterates over each instruction in the update data package and, referring to the update policy matrix, assigns it an execution permission (e.g., "AUTO_EXECUTE" or "MANUAL_REVIEW") and a priority (e.g., 1-5, with 1 being the highest). These permissions and priority information are attached to each instruction in the update data packet to form a hierarchical update scheme.

[0130] All database operations marked "AUTO_EXECUTE" in the hierarchical update plan are output. The system initiates a database transaction. Within the transaction, the system executes these automatically executed database operations in priority order. For example, the system submits the UPDATE statement that updates the Inventory_Main table and the INSERT statement that inserts into the Inventory_State_History table through the database connection pool. The database management system is responsible for executing these operations atomically: if all statements execute successfully, the transaction is committed (COMMIT), and all changes are permanently saved. If any error occurs during execution (for example, a database connection interruption or data conflict), the transaction is rolled back (ROLLBACK), undoing all previous changes and restoring the database to its pre-transaction state. Instructions marked "MANUAL_REVIEW" are not executed at this stage. Instead, a pending task is generated and routed to the manual review workflow. The system records the execution result (success / failure) of each transaction, the execution time, the item IDs involved and the update type, and any errors. These execution records and transaction status information constitute the database update results.

[0131] Preferably, the business process triggering judgment in step S4 includes:

[0132] Perform classification analysis on the database update results to obtain a classification update record table;

[0133] Match the exception handling rules on the classification update record table to obtain the exception handling solution table;

[0134] Generate item transfer instruction set based on exception handling solution table;

[0135] Generate an inventory review task table based on the exception handling plan table;

[0136] Adjust the outbound policy for the quality change records in the classification update record table and obtain the outbound rule modification instruction;

[0137] Generate business process instructions based on item transfer instruction set, inventory review task list and outbound rule modification instructions;

[0138] Integrate business process trigger instructions and database update results to generate warehouse status update instructions.

[0139] In this embodiment of the present invention, these update results are parsed to identify which fields of items have changed, as well as the type of update triggered by the sensor data (e.g., quantity revision, status change, or type discrepancy). This information is organized into a categorized update record table. Based on the detailed change information in the categorized update record table, the system compares it with a pre-set exception handling rule library to identify and match the subsequent business processes that need to be triggered, generating an exception handling plan table. Based on the action items specified in the exception handling plan table, the system generates an item transfer instruction set to move the abnormal items to a designated area. Furthermore, for situations requiring manual intervention and verification, the system generates an inventory review task table to schedule on-site inspections or re-inventory counts. For item quality changes or abnormal status revealed by the sensor data, the system generates an outbound rule modification instruction to adjust the item's priority or availability in the outbound picking process. The system integrates the item transfer instruction set, inventory review task table, and outbound rule modification instruction into one or more business process instruction data packets. Finally, the system merges these business process instruction data packets with the original database update results to form a complete warehouse status update instruction containing inventory change details and subsequent processing instructions.

[0140] Traverse the database update result records. For UPDATE operations on the Inventory_Main table, the system checks the modified fields and identifies the update type: if the Current_Qty field is modified, it is marked as "quantity change"; if the State field is modified, it is marked as "status change"; if the Item_Type field is modified, it is marked as "type change"; if the Last_Inventory_Time field is modified, it is marked as "inventory time update". For INSERT operations on the Inventory_State_History table, it is marked as "abnormal status record added" and a detailed abnormality description and perceived reliability score are extracted from the inserted data. The system organizes the parsed information, including item ID, location ID, update type, specific change content (for example, quantity change value, new status code), and related perceived reliability score, into a structured table to form a classified update record table.

[0141] The system stores an exception handling rule base, for example, the rule : If the update type is "quantity change" and the absolute difference in quantity exceeds 10 or the relative difference exceeds 15% and the perceived reliability score is greater than 0.7, the action "Create inventory review task" is triggered. : If the update type is "status change" and the new status is marked as "serious exception" (for example, code 901 means "liquid container is damaged"), the actions of "generate transfer instructions to the quality inspection area" and "adjust the outbound delivery policy to prohibit outbound delivery" are triggered. Rule R3: If the update type is "type change" and the perception type confidence is lower than 0.85, the action of "create type verification task" is triggered. The system traverses each record in the classification update record table. For each record, the system evaluates all rule conditions in the exception handling rule library. If a record meets the conditions of a rule, the system adds the action defined by the rule (for example, "create inventory review task", "generate transfer instructions", "adjust outbound delivery policy") to the exception handling plan corresponding to the record. If a record meets multiple rules, the corresponding actions are accumulated. The system summarizes all item identifiers and their corresponding list of actions to be performed to form an exception handling plan table.

[0142] The system traverses the exception handling plan table, searching for records containing the "Generate Transfer Instructions" action. For each such record, the system determines the target location to which the item should be transferred (e.g., a pre-defined quality inspection area, scrap area, or pending processing area) based on the exception type or rule definition. By querying the warehouse layout and robot capabilities, the system generates the detailed robot instructions required to execute the transfer operation. The instruction structure includes: task type ("Transfer"), item identifier (Item_ID), current location (Location_ID), target location (Target_Location_ID), transfer reason (e.g., "Serious internal exception detected"), and priority (e.g., high priority). The system collects all items requiring transfer and their corresponding transfer instructions to form an item transfer instruction set. These instructions are then sent to the Warehouse Execution System (WES) or directly to the robot responsible for the transfer.

[0143] The system traverses the exception handling plan table, searching for records containing the "Create Inventory Review Task" action. For each such record, the system generates a review task requiring manual or specialized equipment intervention based on the triggering cause (e.g., large quantity discrepancy, questionable type, low perceived reliability). The task structure includes: task type ("Inventory Review"), item ID (Item_ID), location ID (Location_ID), review content (e.g., "Verify quantity," "Verify item type," "Check internal status"), triggering cause description (e.g., "The perceived estimated quantity differs from the system record by 18%"), task priority, and recommended review method (e.g., manual count, use of specific testing equipment). The system collects information about all items requiring review and their corresponding review tasks to create an inventory review task table. These tasks are then sent to the warehouse operation management system or manual task allocation system.

[0144] Records related to changes in item quality or status are filtered out from the table, such as those with an update type of "status change," new status marked as abnormal, or large quantity changes indicating packaging damage. The system maintains an internal outbound policy management module, which determines item picking priority and availability based on item status, batch, and attributes. Based on filtered quality change records, the system generates instructions to modify the outbound rules for the item. The instruction structure includes the item ID (Item_ID), location ID (Location_ID), new outbound status flag (e.g., "Normal," "Priority Outbound," "Postponed Outbound," "Prohibited Outbound"), and the reason for the adjustment (e.g., "Minor abnormality detected, priority outbound recommended," "Severe abnormality detected, prohibited outbound"). For example, if a liquid container is detected to have minor shaking (but not damage), the system generates an instruction to increase the outbound priority of the batch of items for faster consumption. If severe damage is detected, an instruction is generated to mark the item as prohibited outbound. The system collects all items requiring outbound policy adjustments and their corresponding modification instructions to form an outbound rule modification instruction. These instructions will be sent to the outbound picking system.

[0145] The instructions and tasks for different downstream systems are integrated to generate one or more unified business process instruction data packets. This data packet is a structured message that contains all subsequent business process action items triggered by this perception inventory. For example, a data packet can contain different parts: a list for storing all warehouse transfer instructions, a list for storing all review tasks, and a list for storing all outbound rule modification instructions. The data packet also contains a total task ID, trigger time, and associated perception report ID for tracking. This integrated data structure is the business process instruction. It converts perception results into specific warehouse operations and management tasks.

[0146] The warehouse status update instruction is one of the final outputs of this perception inventory cycle. It not only confirms the update results of the inventory master data (which items have changed in quantity, status, etc.), but also clarifies all subsequent business process actions triggered by it (which items need to be transferred, which need to be reviewed, and which outbound strategies need to be adjusted). The instruction structure includes: a unique instruction identifier; the associated inventory task identifier and perception report identifier; database update details (a list of successfully updated items, updated fields, and new values); triggered business process details (a list of transfer instructions, a list of review tasks, a list of outbound rule modification instructions); and other meta-information (such as generation time and execution system identifier). This comprehensive instruction is sent to the relevant warehouse management subsystems (e.g., WES, TMS, WCS) and monitoring interfaces for subsequent execution, tracking, and manual intervention.

[0147] The present invention is therefore intended to be illustrative and non-restrictive in all respects, with the scope of the invention being defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the application documents are intended to be embraced therein.

[0148] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A warehouse inventory method, characterized in that: The following steps are involved: Step S1: Analyze the inventory demand path of the warehouse to obtain the inventory path plan; issue the inventory task to the robot according to the inventory path plan to obtain the handling task instruction; Step S2: performing inventory item grabbing processing according to the handling task instruction to obtain static contact characteristic data; performing dynamic interaction and multimodal signal collection based on the static contact characteristic data to obtain interactive perception data; Step S3: performing perception signal separation on the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature map; The mechanical parameter feature table and the acoustic feature map are matched to identify and match the item type to obtain an item type determination result. Based on the item type determination result, the internal state of the item is evaluated to obtain an internal state evaluation table. Based on the internal state evaluation table, the quantity / filling amount of items identified as containers in the item type determination result are estimated and a perception report is generated to obtain a material perception feature report. Step S4: Synchronously update the inventory information of the material perception feature report to obtain a database update result; Based on the database update results, the business process trigger judgment is carried out to obtain the warehouse status update instruction; The dynamic interaction and multimodal signal acquisition in step S2 includes: Based on the static contact characteristic data, the grasping force is incrementally controlled to obtain the grasping process characteristic curve; According to the characteristic curve of the grabbing process, the lifting gravity response analysis is carried out to obtain the lifting gravity response curve; Evaluate the object's suspension stability based on the lifting gravity response curve to obtain suspension stability data; Based on the suspension stability data, the perception action is executed and micro-perturbation enhancement is performed to obtain the perturbation action response data; collecting sensor array data based on disturbance action response data; Perform multimodal data time alignment on sensor array data to obtain time-synchronized data streams; Perform dynamic response data integration on the disturbance action response data and the time synchronization data stream to obtain dynamic interaction response data; Performing signal synchronization and noise reduction on the dynamic interactive response data to obtain a noise reduction signal data packet; Encapsulating and transmitting the noise reduction signal data packet to obtain interactive perception data; The quantity / filling amount estimation and perception report generation in step S3 includes: Extracting a filling-related parameter set from an item identified as a container in an item type determination result; Matching the populated relevant parameter set with the pre-established physical model parameter library to obtain the estimation model selection result; Calculate the liquid filling volume based on the estimation model selection result to obtain the calculated value of the liquid filling rate; The particle number is estimated based on the estimation model selection result to obtain the particle filling rate calculation value; Based on the estimation model selection results, solid object count estimation is performed to obtain an estimated value of the number of items; Estimation reliability scores are performed on the liquid filling rate calculation value, the particle filling rate calculation value, and the item quantity estimation value to obtain an estimation reliability score; Calibrate estimated reliability scores with historical data and generate fill volume estimates; A comprehensive report is generated based on the item type determination results, internal state evaluation table, and filling quantity estimation results to obtain a material perception feature report.

2. A warehouse inventory method according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Obtain the current warehouse inventory status data and generate an inventory requirement list according to the preset inventory rules; Step S12: planning the robot's operation path according to the inventory requirement list to obtain an inventory path plan; Step S13: Obtain robot capability data and item characteristic data, and perform robot capability matching according to the inventory path plan to obtain a robot allocation result; Step S14: configuring the robot's perception action according to the robot allocation result to obtain a perception action configuration table; Step S15: Encapsulate and issue the inventory path plan, robot allocation results, and perception action configuration table to obtain the handling task instructions.

3. A warehouse inventory method according to claim 1, characterized in that: The inventory item grabbing process in step S2 includes: Perform command parsing and sensor pre-activation on the handling task instructions to obtain the sensor configuration plan; According to the sensor configuration plan and the storage location information in the handling task instruction, the items in the storage location are approached and positioned to obtain the grasping execution plan; Initial contact and basic characteristics are acquired according to the crawl execution plan to obtain static contact characteristic data.

4. A warehouse inventory method according to claim 1, characterized in that: The sensing signal separation in step S3 includes: Perform signal separation and preprocessing on interactive perception data to obtain classification signal feature set; Extract mechanical characteristic parameters from the classification signal feature set to obtain a mechanical parameter feature table; An acoustic feature map is generated for the classification signal feature set to obtain an acoustic feature map.

5. A warehouse inventory method according to claim 4, characterized in that: Acoustic feature map generation includes: Perform acoustic and vibration time-frequency conversion on the classification signal feature set to obtain the time-frequency energy matrix; Divide the material characteristic frequency bands based on the time-frequency energy matrix to obtain the frequency band statistical characteristic table; Perform spectral centroid and morphology calculations on the time-frequency energy matrix to obtain a spectral morphology parameter sequence; Performing time-varying analysis of the acoustic energy envelope on the classification signal feature set to obtain an envelope dynamic parameter table; Perform acoustic fingerprint matching calculation on the frequency band statistical feature table, spectrum morphology parameter sequence and envelope dynamic parameter table to obtain the material acoustic feature matching table; Extract filling rate acoustic markers based on the material acoustic feature matching table and envelope dynamic parameter table to obtain a filling state acoustic index table; Perform abnormal resonance and acoustic deviation detection on the time-frequency energy matrix to obtain an abnormal acoustic feature table; Generate an acoustic feature map based on the abnormal acoustic feature table and the filling status acoustic index table.

6. A warehouse inventory method according to claim 1, characterized in that: The synchronous update of inventory information in step S4 includes: Receive and verify the material perception characteristic report and obtain verification result records; Compare the inventory data with the material perception feature report based on the verification result records to obtain a data difference analysis table; Update inventory records based on the data difference analysis table to obtain updated data packets; Evaluate the update permissions and priorities of the update data packages to obtain a hierarchical update plan; The database transaction is executed according to the hierarchical update scheme to obtain the database update result.

7. A warehouse inventory method according to claim 1, characterized in that: The business process triggering judgment in step S4 includes: Perform classification analysis on the database update results to obtain a classification update record table; Match the exception handling rules on the classification update record table to obtain the exception handling solution table; Generate item transfer instruction set based on exception handling solution table; Generate an inventory review task table based on the exception handling plan table; Adjust the outbound policy for the quality change records in the classification update record table and obtain the outbound rule modification instruction; Generate business process instructions based on item transfer instruction set, inventory review task list and outbound rule modification instructions; Integrate business process trigger instructions and database update results to generate warehouse status update instructions.

8. A warehousing system, applied to a warehousing inventory method according to any one of claims 1 to 7, characterized in that: The storage system includes: The inventory task dispatching module is used to analyze the inventory demand path of the warehouse and obtain the inventory path plan; according to the inventory path plan, the inventory task is issued to the robot to obtain the handling task instruction; The item interaction perception and acquisition module is used to process inventory items according to handling task instructions to obtain static contact characteristic data; based on the static contact characteristic data, dynamic interaction and multimodal signal acquisition are performed to obtain interaction perception data; The material feature intelligent analysis module is used to separate the perception signals of the interactive perception data to obtain a mechanical parameter feature table and an acoustic feature spectrum; perform item type identification and matching on the mechanical parameter feature table and the acoustic feature spectrum to obtain an item type determination result; perform an internal status assessment of the item based on the item type determination result to obtain an internal status assessment table; and perform quantity / filling volume estimation and perception report generation for items identified as containers in the item type determination result based on the internal status assessment table to obtain a material perception feature report; The inventory status update and trigger module is used to synchronize the inventory information of the material perception feature report to obtain the database update result; based on the database update result, the business process trigger judgment is performed to obtain the warehouse status update instruction.

Images