A flexible, organized, intelligent automated production system for unordered incoming materials
By incorporating the physical state changes and dynamic spatiotemporal constraints of surrounding materials in real time into the unordered incoming material flexible and orderly intelligent automated production system, the system generates fine-tuning target pose and optimal grasping time, solving the problems of prediction failure and process blockage in existing technologies, and realizing stable continuous flow and stable cycle time of the production line.
Patent Information
- Application Number
- CN202611285630.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-24
- Publication Date
- 2026-11-17
AI Technical Summary
The existing unordered incoming material flexible and orderly intelligent automated production system does not incorporate the changes in the physical state of surrounding materials caused by the execution of actions and the occupancy status of orderly workstations into the dynamic calculation closed loop during scheduling. This leads to prediction failure, trajectory conflicts and frequent global replanning, causing chain blockage and deadlock in the process, and failing to meet the stability requirements of continuous production.
By acquiring initial environmental image data, extracting material contour edge data and center of gravity coordinate data, calculating motion trajectory envelope data, establishing a spatiotemporal sliding window, incorporating changes in the physical state of surrounding materials and dynamic spatiotemporal constraints in real time, generating fine-tuned target pose and optimal grasping time, and iteratively executing the process to avoid prediction failure and process blockage.
It enables real-time incorporation of changes in the physical state of surrounding materials into the scheduling closed loop, eliminating prediction failures and global replanning, ensuring the continuous flow of the regular process and the overall cycle stability of the production line, and avoiding deadlock.
Smart Images

Figure CN123103434A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automation control technology, and in particular to a flexible and orderly intelligent automated production system for disordered incoming materials. Background Technology
[0002] With the deepening application of flexible manufacturing in industrial production, automated production lines face increasingly stringent requirements for the adaptive recognition and flexible arrangement capabilities of incoming materials, which are often multi-category and randomly stacked. Existing intelligent automated production systems for flexible arrangement of unordered incoming materials acquire instantaneous global snapshots through vision systems and generate grasping sequences for actuators and isolated path planning with the shortest single path as the objective to achieve material grasping and placement based on these snapshots.
[0003] In existing technologies, the scheduling methods based on instantaneous static snapshots and isolated optimization do not incorporate the changes in the physical state of surrounding materials caused by the execution action and the timing occupancy status of the regularized workstations into the dynamic calculation closed loop. This results in problems such as scheduling benchmark lag leading to prediction failure and local path accumulation causing regularized resource congestion. Due to prediction failure and resource congestion, the execution mechanism is prone to capture failure or trajectory spatiotemporal overlap conflicts during movement, which in turn frequently triggers global image re-acquisition and path replanning, causing chain-like blockages and redundant idling. In this case, the production process needs to continuously perform global state resets and post-conflict compensation calculations, resulting in an uncontrollable lengthening of the overall cycle time of flexible regularization. In multi-mechanism collaboration and compact high-cycle scenarios, resource dependency loop waiting deadlocks and limited scenario compatibility are likely to occur, ultimately causing the cycle time of the automated production line to collapse and failing to meet the stability requirements of continuous production. Summary of the Invention
[0004] This application provides a flexible and orderly intelligent automated production system for unordered incoming materials. It solves the problems in the prior art where scheduling logic based on instantaneous static snapshots and isolated optimization does not incorporate execution disturbances and resource occupancy status into the dynamic calculation closed loop, leading to prediction failure, trajectory conflicts, and chain blockages and deadlocks caused by frequent global replanning. The system incorporates the changes in the physical state of surrounding materials and dynamic spatiotemporal constraints into the scheduling closed loop in real time. By dynamically fine-tuning the target pose and rearranging the sequence, global replanning is eliminated, thereby ensuring the continuous flow of the orderly process and completely eliminating deadlocks to maintain the overall stable cycle time of the automated production line.
[0005] This application provides a flexible and orderly intelligent automated production system for unordered incoming materials, including: a data acquisition module: used to acquire initial environmental image data, extract the contour edge data, center of gravity coordinate data and stacking correlation data of all materials, determine the first grab target material and generate the first grab instruction;
[0006] Motion trajectory envelope data acquisition module: used to extract the surrounding related materials that have a contact boundary with the first grasp target material during the execution of the first grasp instruction, calculate the displacement trend and spatial range of the surrounding related materials, and obtain motion trajectory envelope data;
[0007] Pose coordinate calculation module: used to establish a spatiotemporal sliding window, bind the motion trajectory envelope data with the time slice within a continuous time slice, and calculate the theoretical pose coordinates of the material to be grasped in the corresponding time slice.
[0008] The optimal grasping time generation module is used to compare the initial identification pose of the subsequent material to be grasped with the theoretical pose coordinates to obtain the pose deviation value. When the pose deviation value exceeds the preset tolerance, the offset compensation amount and the stabilization recovery time are calculated to generate the fine-tuning target pose and the optimal grasping time.
[0009] Instruction execution module: used to encapsulate the fine-tuned target pose and the optimal grasping time into a new grasping instruction and issue it for execution, iterating in a loop until all materials are grasped.
[0010] Furthermore, the steps of determining the target material for the first grab and generating the first grab instruction include:
[0011] Feature extraction and coordinate transformation are performed on the initial environmental image data to obtain the contour edge data, centroid coordinate data, and stacking correlation data between materials for each material;
[0012] Retrieve data on the exposed area ratio, center of gravity height offset, and straight-line distance from the gripping position for each individual material;
[0013] The exposure area ratio, center of gravity height offset value, and straight-line distance data are input into the pre-set priority calculation rules, and a weighted calculation is performed to obtain a comprehensive score.
[0014] Set the material with the highest overall score as the first target material to grab;
[0015] By combining the location data of the target material to be grabbed with the motion parameters of the mechanical gripper, the initial path coordinates and time nodes are calculated, and the first grab command is synthesized.
[0016] Further, the step of calculating the displacement trend and spatial range of the surrounding related materials to obtain the motion trajectory envelope data includes:
[0017] Extract the outline edge data of the first target material to be grabbed, traverse the entire incoming material area data set, and retrieve the surrounding related material records that are in contact with or stacked with the first target material to be grabbed.
[0018] Retrieve the center of gravity coordinates and contact boundary contour data of the first target material and surrounding related materials;
[0019] Based on the rigid body motion force analysis conditions, calculate the displacement direction of the surrounding related materials when the target material is lifted upward and detached from the placement plane.
[0020] Calculate the sliding probability and overturning risk caused by changes in contact friction.
[0021] By combining the displacement direction, sliding probability value, and overturning risk value, the distribution data of the movement range of surrounding related materials in three-dimensional space can be calculated.
[0022] Furthermore, calculating the displacement trend and spatial range of surrounding materials to obtain the motion trajectory envelope data also includes:
[0023] Obtain data on the displacement direction and spatial distribution of surrounding materials after being physically affected;
[0024] Define discretization rules for the three-dimensional spatial coordinate system to break down the spatially distributed data into interconnected spatial grid particles along the horizontal, vertical, and height directions;
[0025] Calculate the probability that each spatial grid particle will be occupied by material at different times;
[0026] Filter out spatial grid particles with a non-zero probability and combine them to generate a dataset containing grid spatial coordinates;
[0027] The dataset is defined as the motion trajectory envelope data of surrounding related materials, which serves as the dynamic boundary constraint for subsequent route planning.
[0028] Furthermore, the steps for calculating the theoretical pose coordinates of the material to be grabbed in the corresponding time slice include:
[0029] Construct a three-dimensional spatiotemporal sliding window covering both the disordered material receiving area and the regular area, and keep the window's timeline synchronized with the time point when the first grab command is issued;
[0030] Divide the timeline into continuous time slices of fixed duration.
[0031] Within each time slice unit, retrieve the motion trajectory envelope data for the corresponding time period, and establish a mapping and binding relationship between the grid spatial coordinates and the time slice unit;
[0032] Based on the mapping and binding relationship, the theoretical executable position and attitude coordinates of the material to be grabbed are calculated after being affected by the sliding of surrounding materials in a specific time slice unit.
[0033] Furthermore, the steps for generating the fine-tuned target pose and optimal grasping moment include:
[0034] The initial recognized pose data is subtracted from the theoretical executable position and attitude coordinates in the current time slice unit to obtain the three-dimensional spatial deviation value.
[0035] When the value of the three-dimensional spatial deviation is determined to be greater than the preset safety tolerance limit, dynamic compensation of the motion trajectory is triggered.
[0036] Based on the displacement distance and direction of movement of the surrounding materials, the required offset compensation values of the gripping end component in each coordinate axis direction are calculated.
[0037] The initial trajectory endpoint coordinates and offset compensation values are vector-superimposed to generate the fine-tuned target pose coordinates.
[0038] Furthermore, the steps for generating the fine-tuned target pose and optimal grasping time also include:
[0039] Retrieve the data on the decreasing trend of displacement of surrounding materials in space over time, which is contained in the motion trajectory envelope data.
[0040] The decay process of surrounding materials returning from an unstable and swaying state to a statically stable state is analyzed, and the time point at which surrounding materials become completely still is predicted.
[0041] Based on the point in time when everything is completely still, the most suitable start time for grabbing subsequent materials can be calculated.
[0042] The original planned capture sequence timeline was postponed or advanced according to the most suitable capture start time;
[0043] Determine the optimal gripping time by starting the gripping action when it avoids the period when surrounding materials collide or tumble.
[0044] Furthermore, the steps of encapsulating the fine-tuning of the target pose and the optimal grasping moment into a new grasping command and issuing it for execution include:
[0045] Extract and merge the fine-tuned target pose coordinate data and the optimal capture time data to generate a capture execution data package containing three-dimensional spatial coordinate values and time point control information.
[0046] The captured execution data packets are converted into data formats according to the communication protocol requirements and encapsulated to generate new capture and execution instructions that are adapted to the execution mechanism interface.
[0047] A new grabbing and execution instruction is issued to the execution agency, driving the execution agency to perform grabbing actions according to the new time point and coordinate position;
[0048] Throughout the entire process of issuing new grabbing and execution instructions and physical grabbing and execution, the original calculation data is maintained, and the triggering of instructions to re-acquire images and identify features in disordered incoming material areas is blocked.
[0049] Furthermore, the iterative process until all materials have been grabbed includes:
[0050] After completing the physical grabbing and detachment of the current first target material, the material is cleaned up and deleted from the data set to be grabbed.
[0051] Update the current operational status of the executing agency to "awaiting readiness";
[0052] Extract the material data of the next sequential position from the records in the dataset to be crawled, and update it with the new subsequent material to be crawled;
[0053] Repeatedly run disturbance prediction, trajectory envelope generation, spatiotemporal window calculation, and capture time series rearrangement operations;
[0054] The physical motion data of the surrounding materials is updated in real time with each grabbing and detaching action, which serves as a constraint on the next material grabbing route until all materials recorded in the disordered material receiving area have been grabbed and transferred.
[0055] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0056] Existing unordered material handling methods fail to incorporate changes in the physical state of surrounding materials caused by the grasping action into the dynamic calculation loop during scheduling, leading to a time phase difference between the issued grasping target pose and the actual physical state of the materials, thus causing prediction failure. This paper addresses this issue by simultaneously extracting surrounding materials and calculating their displacement trends and trajectory envelope data during the execution of the first grasping command. Combined with a spatiotemporal sliding window to deduce theoretical pose coordinates, the changes in the physical state of surrounding materials are incorporated as dynamic constraints into the scheduling logic in real time. This eliminates the time phase difference between the issued pose and the actual state, effectively preventing prediction failure.
[0057] Furthermore, in eliminating prediction failures, considering that missing the target or attitude interference would trigger a global re-acquisition of images and replanning of the path, leading to single-arm occupancy waiting and chain-like blockages, the system compares the initial recognized pose with the theoretical pose coordinates. When the deviation exceeds the tolerance, it directly calculates the offset compensation and stabilization recovery time, generates a fine-tuned target pose and the optimal grasping time, and sends them out. Throughout this process, it shields the triggering of commands to re-acquire images and re-identify features. This locally adaptive trajectory fine-tuning and timing rearrangement mechanism allows the actuator to autonomously adjust grasping parameters and avoid risky periods when faced with disturbances from surrounding materials. This avoids global replanning triggered by grasping failures, ensures the continuous flow of the orderly process, and prevents chain-like blockages.
[0058] Furthermore, in avoiding chain-like blockages and ensuring continuous process flow, it's important to consider that frequent local adjustments, if not updated with the actual state, could lead to task backlog and resource dependency loops causing deadlocks, ultimately resulting in system cycle time collapse. This is addressed by iteratively adjusting the target pose and optimal grabbing moment, and updating the physical motion simulation data of surrounding materials in real time after each material grabbing and detachment action as a constraint for the next material grabbing route. This closed-loop rolling simulation based on actual execution actions enables the production process to continuously output collision-free grabbing instructions without global reset, completely eliminating the time consumption caused by global state resets and path recalculations. It avoids the backlog of tasks and deadlocks caused by multiple interdependent execution mechanisms, ensuring a flexible and regular overall cycle time and the stability of continuous production. Attached Figure Description
[0059] Figure 1 This application provides a structural schematic diagram of a flexible, orderly, intelligent automated production system for unordered incoming materials. Detailed Implementation
[0060] This application provides a flexible and orderly intelligent automated production system for unordered incoming materials. It solves the problems in the prior art where scheduling benchmark lag leads to motion prediction failure, chain replanning causes process blockage and deadlock leading to production line cycle time collapse. By extracting the motion trajectory envelope data of surrounding related materials during the execution of the first grab instruction, establishing a spatiotemporal sliding window to deduce the theoretical pose coordinates, and comparing the initial pose with the theoretical pose to calculate the offset compensation amount and stabilization recovery time, it generates the fine-tuning target pose and the optimal grab time, and iteratively issues them for execution. This achieves the effect of dynamically incorporating the physical disturbance of surrounding materials into the scheduling closed loop to eliminate the pose time phase difference, eliminate global replanning to avoid chain blockage, and completely eliminate deadlock to maintain the overall cycle time stability of the automated production line.
[0061] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.
[0062] like Figure 1 The diagram shown is a structural schematic of a disordered incoming material flexible and orderly intelligent automated production system provided in an embodiment of this application, including: a data acquisition module: used to acquire initial environmental image data, extract the contour edge data, center of gravity coordinate data and stacking association data of all materials, determine the first grab target material and generate the first grab instruction;
[0063] Motion trajectory envelope data acquisition module: During the execution of the first grasping instruction, it is used to extract the surrounding related materials that have contact boundaries with the first grasping target material, calculate the displacement trend and spatial range of the surrounding related materials based on contact constraints and gravity, and obtain motion trajectory envelope data;
[0064] Pose coordinate calculation module: used to establish a spatiotemporal sliding window, bind the motion trajectory envelope data with the time slice within a continuous time slice, and calculate the theoretical pose coordinates of the material to be grasped in the corresponding time slice.
[0065] The optimal grasping time generation module is used to compare the initial identification pose of the subsequent material to be grasped with the theoretical pose coordinates to obtain the pose deviation value. When the pose deviation value exceeds the preset tolerance, the offset compensation amount and the stabilization recovery time are calculated to generate the fine-tuning target pose and the optimal grasping time.
[0066] Instruction execution module: used to encapsulate the fine-tuned target pose and the optimal grasping time into a new grasping instruction and issue it for execution, iterating in a loop until all materials are grasped.
[0067] Furthermore, the steps of determining the target material for the first grab and generating the first grab instruction include:
[0068] Feature extraction and coordinate transformation are performed on the initial environmental image data to obtain the contour edge data, centroid coordinate data, and stacking correlation data between materials for each material;
[0069] Retrieve data on the exposed area ratio, center of gravity height offset, and straight-line distance from the gripping position for each individual material;
[0070] The exposure area ratio, center of gravity height offset value, and straight-line distance data are input into the pre-set priority calculation rules, and a weighted calculation is performed to obtain a comprehensive score.
[0071] Set the material with the highest overall score as the first target material to grab;
[0072] By combining the location data of the target material to be grabbed with the motion parameters of the mechanical gripper, the initial path coordinates and time nodes are calculated, and the first grab command is synthesized.
[0073] In this embodiment, a multi-dimensional weighted scoring logic is used to achieve efficient screening of primary targets, specifically including the following steps:
[0074] The initial environmental image data was segmented, edge detected, and coordinate system transformed to extract the contour edge data and centroid 3D coordinate data of each independent material in the disordered incoming material area. And data on the stacking and occlusion relationships between materials.
[0075] For each identified material, retrieve its exposed area ratio data. (The ratio of the unobstructed visual projection area of the material to the actual full area of the material, ranging from [0,1]), and the value of the center of gravity height offset. And the spatial straight-line distance data between the center of gravity coordinates of the material and the origin of the actuator's base coordinates. .
[0076] Substitute the above data into the pre-set priority calculation formula for comprehensive scoring, and the priority comprehensive score is calculated. The calculation formula is as follows:
[0077] ;
[0078] In the formula, For the first The overall priority score of each material to be grabbed (the higher the value, the higher the grab priority). For the first The current actual exposed area of each material; This represents the maximum exposed area among all currently identified materials, used to normalize the exposed area to [0,1]. For the first The center of gravity of each material Axis height coordinates; The lowest and highest points of gravity of all materials in the current region The axis height coordinate is used to normalize the height to [0,1]. For the first The three-dimensional Euclidean distance between the center of gravity of each material and the origin of the actuator's base coordinate system is calculated using the following formula: ; : The maximum distance from all materials within the current area to the origin of the base coordinate system; The preset weighting coefficients satisfy... In this embodiment, the default settings are... , , .
[0079] Compare the overall priority scores of all materials The material with the largest value is selected as the first target material to be grabbed.
[0080] By combining the three-dimensional pose coordinates of the target material to be grasped with the kinematic constraint parameters of the actuator, an initial approach path and time node control coordinates are generated, and the initial grasp command is synthesized and issued.
[0081] Further, the step of calculating the displacement trend and spatial range of the surrounding related materials to obtain the motion trajectory envelope data includes:
[0082] Extract the outline edge data of the first target material to be grabbed, traverse the entire incoming material area data set, and retrieve the surrounding related material records that are in contact with or stacked with the first target material to be grabbed.
[0083] Retrieve the center of gravity coordinates and contact boundary contour data of the first target material and surrounding related materials;
[0084] Based on the rigid body motion force analysis conditions, calculate the displacement direction of the surrounding related materials when the target material is lifted upward and detached from the placement plane.
[0085] Calculate the sliding probability and overturning risk caused by changes in contact friction.
[0086] By combining the displacement direction, sliding probability value, and overturning risk value, the distribution data of the movement range of surrounding related materials in three-dimensional space can be calculated.
[0087] In this embodiment, a force calculation model based on rigid body statics and kinematics is established, which specifically includes the following steps:
[0088] Extract the contour edge data of the initial target material, and search through all materials within the region using a 3D bounding box and edge spacing. Select materials whose boundary distance from the initial target material is less than a preset contact threshold (e.g., ...). Materials that are located below the target material being grasped are marked as related materials and their center of gravity coordinates and contact surface contour data are retrieved.
[0089] Calculate the instantaneous changes in the supporting force and frictional force on the surrounding materials when the initial grasped target material is lifted upwards and completely detaches from the contact surface. Based on the static equilibrium condition of a rigid body, calculate the probability of the surrounding materials sliding along the inclined plane after losing their original supporting force. The calculation formula is as follows:
[0090] ;
[0091] In the formula, The probability of slippage instability caused by surrounding materials, with a value range of [0,1]. Estimated quality of surrounding related materials; Let gravitational acceleration be the value. ; The inclination angle of the supporting contact surface where the surrounding related materials are located; The sliding friction coefficient between material surfaces is typically 0.2 to 0.6 (based on the material properties). For extremely small positive real numbers (such as This is used to prevent calculation errors caused by a denominator of zero.
[0092] Meanwhile, the overturning risk value of surrounding related materials. The calculation is as follows:
[0093] ;
[0094] in, The vertical height of the center of gravity of the surrounding materials from the overturning fulcrum; It is the minimum horizontal distance from the center of gravity projection point to the overturning edge.
[0095] Combined with displacement direction vector Sliding probability Overturning risk value The three-dimensional spatial probability displacement distribution range of surrounding related materials after the physical constraints are removed is calculated by comprehensively estimating.
[0096] Furthermore, calculating the displacement trend and spatial range of surrounding materials to obtain the motion trajectory envelope data also includes:
[0097] Obtain data on the displacement direction and spatial distribution of surrounding materials after being physically affected;
[0098] Define discretization rules for the three-dimensional spatial coordinate system to break down the spatially distributed data into interconnected spatial grid particles along the horizontal, vertical, and height directions;
[0099] Calculate the probability that each spatial grid particle will be occupied by material at different times;
[0100] Filter out spatial grid particles with a non-zero probability and combine them to generate a dataset containing grid spatial coordinates;
[0101] The dataset is defined as the motion trajectory envelope data of surrounding related materials, which serves as the dynamic boundary constraint for subsequent route planning.
[0102] In this embodiment, a spatial grid discretization method is used in the process of generating motion trajectory envelope data, specifically including the following steps:
[0103] Obtain the calculated three-dimensional spatial probability displacement distribution range of surrounding related materials after physical disturbance.
[0104] Define the discretization rules for the three-dimensional spatial coordinate system, and divide the three-dimensional space containing the disordered incoming material region along... axis, shaft and The axial direction is cut into interconnected cubic spatial grid particles, and the side length of each spatial grid particle is [dimension missing]. Set as .
[0105] Calculate each spatial grid particle In time The probability value of being occupied by surrounding related physical entities. .
[0106] Set an occupancy probability filtering threshold (e.g., 0.05) to filter out all grid particles with an occupancy probability greater than zero, extract their 3D spatial center coordinates, and combine them to generate a dataset containing the grid spatial coordinates. :
[0107] ;
[0108] This data set Defined as the motion trajectory envelope data of surrounding related materials, it serves as a dynamic no-entry boundary condition for subsequent route planning of materials to be captured.
[0109] Furthermore, the steps for calculating the theoretical pose coordinates of the material to be grabbed in the corresponding time slice include:
[0110] Construct a three-dimensional spatiotemporal sliding window covering both the disordered material receiving area and the regular area, and keep the window's timeline synchronized with the time point when the first grab command is issued;
[0111] Divide the timeline into continuous time slices of fixed duration.
[0112] Within each time slice unit, retrieve the motion trajectory envelope data for the corresponding time period, and establish a mapping and binding relationship between the grid spatial coordinates and the time slice unit;
[0113] Based on the mapping and binding relationship, the theoretical executable position and attitude coordinates of the material to be grabbed are calculated after being affected by the sliding of surrounding materials in a specific time slice unit.
[0114] In this embodiment, the spatiotemporal mapping calculation is implemented during the process of establishing a spatiotemporal sliding window and calculating the theoretical pose coordinates of the material to be grasped in the corresponding time slice. This includes the following steps:
[0115] Establish a three-dimensional spatiotemporal sliding window covering the entire disordered incoming material area and the ordered area, and set the starting and ending points of the sliding window in time. It is synchronized with the absolute timestamp of the first capture command.
[0116] Divide the timeline into consecutive time slices of equal length according to a fixed duration. In this embodiment, it is set .
[0117] In each time slice unit ( Within this range, retrieve the motion trajectory envelope data corresponding to that time period. And the grid spatial coordinates in the envelope data are compared with the time slice units. Bind.
[0118] Based on the changes in the bound spatial grid, the subsequent material to be grabbed will be estimated in a specific time slice unit. Theoretically executable pose coordinates after being pushed, slid, or blocked by surrounding materials .
[0119] Furthermore, the steps for generating the fine-tuned target pose and optimal grasping moment include:
[0120] The initial recognized pose data is subtracted from the theoretical executable position and attitude coordinates in the current time slice unit to obtain the three-dimensional spatial deviation value.
[0121] When the value of the three-dimensional spatial deviation is determined to be greater than the preset safety tolerance limit, dynamic compensation of the motion trajectory is triggered.
[0122] Based on the displacement distance and direction of movement of the surrounding materials, the required offset compensation values of the gripping end component in each coordinate axis direction are calculated.
[0123] The initial trajectory endpoint coordinates and offset compensation values are vector-superimposed to generate the fine-tuned target pose coordinates.
[0124] In this embodiment, during the specific implementation of the steps of calculating the deviation and generating the fine-tuned target pose, the system completes the path fine-tuning through vector superposition, specifically including the following steps:
[0125] Obtain the initial recognition pose coordinates of the material to be grasped during the initial visual acquisition phase. The theoretical executable pose coordinates under the current time slice unit. Perform vector subtraction to calculate the three-dimensional spatial position deviation vector. With attitude angle deviation vector :
[0126]
[0127]
[0128] Calculate the Euclidean distance modulus of the three-dimensional spatial position deviation
[0129] and compare it with the preset safety tolerance limit value. (This embodiment is set as follows) Perform a size comparison.
[0130] When the judgment At this time, the dynamic compensation logic for the motion trajectory is triggered. Based on the displacement direction and distance of surrounding materials, the required offset compensation vector for the grasping end effector in each coordinate axis direction is calculated. .
[0131] The endpoint coordinates of the initial approach trajectory The offset compensation vector is then superimposed to generate the fine-tuned target pose coordinates. :
[0132] .
[0133] Furthermore, the steps for generating the fine-tuned target pose and optimal grasping time also include:
[0134] Retrieve the data on the decreasing trend of displacement of surrounding materials in space over time, which is contained in the motion trajectory envelope data.
[0135] The decay process of surrounding materials returning from an unstable and swaying state to a statically stable state is analyzed, and the time point at which surrounding materials become completely still is predicted.
[0136] Based on the point in time when everything is completely still, the most suitable start time for grabbing subsequent materials can be calculated.
[0137] The original planned capture sequence timeline was postponed or advanced according to the most suitable capture start time;
[0138] Determine the optimal gripping time by starting the gripping action when it avoids the period when surrounding materials collide or tumble.
[0139] In this embodiment, the process of calculating the stable recovery time and generating the optimal grasping moment employs a motion energy decay estimation model, specifically including the following steps:
[0140] Retrieve the decay trend data of the displacement amplitude of surrounding materials in space over time from the motion trajectory envelope data. Motion amplitude decay function. The model is as follows:
[0141] ;
[0142] In the formula, In time The residual sway amplitude of surrounding materials at that time. This represents the initial disturbance amplitude at the moment when the surrounding materials become unstable. This is the material damping coefficient, with a value ranging from 0.5 to 3.0 (set according to the surface friction and elastic coefficient of the material). This is the starting point at which the physical disturbance begins.
[0143] When the residual sway amplitude Decay to the safe rest threshold (like When the following conditions are met, the expected recovery time for the surrounding materials to return to a statically stable state can be calculated. :
[0144] ;
[0145] Based on the expected static time node Add to the preset safety buffer zone (like ), calculate the optimal start time for grabbing subsequent materials. .
[0146] Compared to the original grabbing timeline, the grabbing sequence of subsequent materials to be grabbed is postponed or advanced accordingly, so that the physical grabbing action occurs in a way that avoids the high-risk period of violent collision and tumbling of surrounding materials.
[0147] Furthermore, the steps of encapsulating the fine-tuning of the target pose and the optimal grasping moment into a new grasping command and issuing it for execution include:
[0148] Extract and merge the fine-tuned target pose coordinate data and the optimal capture time data to generate a capture execution data package containing three-dimensional spatial coordinate values and time point control information.
[0149] The captured execution data packets are converted into data formats according to the communication protocol requirements and encapsulated to generate new capture and execution instructions that are adapted to the execution mechanism interface.
[0150] A new grabbing and execution instruction is issued to the execution agency, driving the execution agency to perform grabbing actions according to the new time point and coordinate position;
[0151] Throughout the entire process of issuing new grabbing and execution instructions and physical grabbing and execution, the original calculation data is maintained, and the triggering of instructions to re-acquire images and identify features in disordered incoming material areas is blocked.
[0152] In this embodiment, the process of encapsulating the fine-tuning of the target pose and the optimal grasping moment into a new grasping instruction and issuing it for execution achieves direct instruction-driven operation with zero image latency, specifically including the following steps:
[0153] Extract the fine-tuned target pose coordinate data With the best start time of the grab This generates a complete capture execution data structure that includes spatial three-dimensional target points, attitude quaternions, and time control information.
[0154] The grabbing execution data structure is serialized according to the communication protocol of the underlying motion controller (such as industrial Ethernet communication protocol, Modbus protocol or custom binary data packet), and packaged and encapsulated to generate a new grabbing execution instruction adapted to the actuator interface.
[0155] The newly generated grabbing and execution instructions are directly sent to the drive queue of the underlying execution mechanism, which then accurately completes the physical grabbing according to the set optimal grabbing start time and the fine-tuned target coordinates.
[0156] Throughout the entire lifecycle of this instruction issuance and physical grasp execution, continuous access to algorithm-deduced data is maintained, completely shielding and intercepting signal instructions that trigger global image re-acquisition and feature recognition in disordered incoming material areas, thus avoiding emergency stop waiting and process interruption caused by image re-acquisition.
[0157] Furthermore, the iterative process until all materials have been grabbed includes:
[0158] After completing the physical grabbing and detachment of the current first target material, the material is cleaned up and deleted from the data set to be grabbed.
[0159] Update the current operational status of the executing agency to "awaiting readiness";
[0160] Extract the material data of the next sequential position from the records in the dataset to be crawled, and update it with the new subsequent material to be crawled;
[0161] Repeatedly run disturbance prediction, trajectory envelope generation, spatiotemporal window calculation, and capture time series rearrangement operations;
[0162] The physical motion data of the surrounding materials is updated in real time with each grabbing and detaching action, which serves as a constraint on the next material grabbing route until all materials recorded in the disordered material receiving area have been grabbed and transferred.
[0163] In this embodiment, the entire process of data flow and iterative control is achieved during the iterative process until all materials are grabbed, specifically including the following steps:
[0164] Once the underlying actuator reports that it has completed the physical grabbing of the current first target material and successfully lifted it away from the disordered material receiving area, the system physically removes the unique identifier of the material from the data set to be grabbed.
[0165] Update the status of the executor recorded in the scheduling system to the idle waiting state (ReadyState).
[0166] Extract the next priority material data from the remaining data set to be captured, sorted by priority, and update it as the new material to be captured.
[0167] Using the latest physical state deduction result after the current material is grasped and detached as the pre-constraint, the steps of predicting disturbances of surrounding related materials, generating three-dimensional spatial trajectory envelope, calculating the theoretical pose of the spatiotemporal sliding window, dynamic trajectory compensation, and rearranging the grasping sequence are repeated.
[0168] This logic is continuously iterated, with each grabbing action updating the dynamic deduction constraints of the remaining materials in real time, until the set of data to be grabbed is empty, that is, all materials in the disordered material area have been successfully grabbed and transferred to the orderly area, completing the entire flexible automated production orderly process.
[0169] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.
[0170] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.
[0171] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0172] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0173] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0174] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A flexible structured smart automated production system for disordered incoming materials, characterized in that, include: Data acquisition module: used to acquire initial environmental image data, extract the contour edge data, centroid coordinate data and stacking correlation data of all materials, determine the first grab target material and generate the first grab instruction; Motion trajectory envelope data acquisition module: used to extract the surrounding related materials that have a contact boundary with the first grasp target material during the execution of the first grasp instruction, calculate the displacement trend and spatial range of the surrounding related materials, and obtain motion trajectory envelope data; Pose coordinate calculation module: used to establish a spatiotemporal sliding window, bind the motion trajectory envelope data with the time slice within a continuous time slice, and calculate the theoretical pose coordinates of the material to be grasped in the corresponding time slice. The optimal grasping time generation module is used to compare the initial identification pose of the subsequent material to be grasped with the theoretical pose coordinates to obtain the pose deviation value. When the pose deviation value exceeds the preset tolerance, the offset compensation amount and the stabilization recovery time are calculated to generate the fine-tuning target pose and the optimal grasping time. Instruction execution module: used to encapsulate the fine-tuned target pose and the optimal grasping time into a new grasping instruction and issue it for execution, iterating in a loop until all materials are grasped.
2. The disordered flexible regular intelligent automated production system of claim 1, wherein, The steps for determining the target material for the first grab and generating the first grab instruction include: Feature extraction and coordinate transformation are performed on the initial environmental image data to obtain the contour edge data, centroid coordinate data, and stacking correlation data between materials for each material; Retrieve data on the exposed area ratio, center of gravity height offset, and straight-line distance from the gripping position for each individual material; The exposure area ratio, center of gravity height offset value, and straight-line distance data are input into the pre-set priority calculation rules, and a weighted calculation is performed to obtain a comprehensive score. Set the material with the highest overall score as the first target material to grab; By combining the location data of the target material to be grabbed with the motion parameters of the mechanical gripper, the initial path coordinates and time nodes are calculated, and the first grab command is synthesized.
3. The flexible structured smart automated production system of claim 1, wherein, The steps for calculating the displacement trend and spatial range of the surrounding related materials to obtain the motion trajectory envelope data include: Extract the outline edge data of the first target material to be grabbed, traverse the entire incoming material area data set, and retrieve the surrounding related material records that are in contact with or stacked with the first target material to be grabbed. Retrieve the center of gravity coordinates and contact boundary contour data of the first target material and surrounding related materials; Based on the rigid body motion force analysis conditions, calculate the displacement direction of the surrounding related materials when the target material is lifted upward and detached from the placement plane. Calculate the sliding probability and overturning risk caused by changes in contact friction. By combining the displacement direction, sliding probability value, and overturning risk value, the distribution data of the movement range of surrounding related materials in three-dimensional space can be calculated.
4. The flexible structured smart automated production system of claim 1, wherein, Calculating the displacement trend and spatial range of surrounding materials to obtain motion trajectory envelope data also includes: Obtain data on the displacement direction and spatial distribution of surrounding materials after being physically affected; Define discretization rules for the three-dimensional spatial coordinate system to break down the spatially distributed data into interconnected spatial grid particles along the horizontal, vertical, and height directions; Calculate the probability that each spatial grid particle will be occupied by material at different times; Filter out spatial grid particles with a non-zero probability and combine them to generate a dataset containing grid spatial coordinates; The dataset is defined as the motion trajectory envelope data of surrounding related materials, which serves as the dynamic boundary constraint for subsequent route planning.
5. The flexible structured smart automated production system of claim 1, wherein, The steps for calculating the theoretical pose coordinates of the material to be grabbed in the corresponding time slice include: Construct a three-dimensional spatiotemporal sliding window covering both the disordered material receiving area and the regular area, and keep the window's timeline synchronized with the time point when the first grab command is issued; Divide the timeline into continuous time slices of fixed duration. Within each time slice unit, retrieve the motion trajectory envelope data for the corresponding time period, and establish a mapping and binding relationship between the grid spatial coordinates and the time slice unit; Based on the mapping and binding relationship, the theoretical executable position and attitude coordinates of the material to be grabbed are calculated after being affected by the sliding of surrounding materials in a specific time slice unit.
6. The flexible structured smart automated production system of claim 1, wherein, The steps for generating the fine-tuned target pose and optimal grasping moment include: The initial recognized pose data is subtracted from the theoretical executable position and attitude coordinates in the current time slice unit to obtain the three-dimensional spatial deviation value. When the value of the three-dimensional spatial deviation is determined to be greater than the preset safety tolerance limit, dynamic compensation of the motion trajectory is triggered. Based on the displacement distance and direction of movement of the surrounding materials, the required offset compensation values of the gripping end component in each coordinate axis direction are calculated. The initial trajectory endpoint coordinates and offset compensation values are vector-superimposed to generate the fine-tuned target pose coordinates.
7. The flexible structured smart automated production system of claim 1, wherein, The steps for generating the fine-tuned target pose and the optimal grasping moment also include: Retrieve the decay trend data of the spatial displacement of surrounding materials contained in the motion trajectory envelope data, which gradually decreases over time; The decay process of surrounding materials returning from an unstable and swaying state to a statically stable state is analyzed, and the time point at which surrounding materials become completely still is predicted. Based on the point in time when everything is completely still, the most suitable start time for grabbing subsequent materials can be calculated. The original planned capture sequence timeline was postponed or advanced according to the most suitable capture start time; Determine the optimal gripping time by starting the gripping action when it avoids the period when surrounding materials collide or tumble.
8. The flexible structured smart automated production system of claim 1, wherein, The steps of encapsulating the fine-tuning of the target pose and the optimal grasping moment into a new grasping command and issuing it for execution include: Extract and merge the fine-tuned target pose coordinate data and the optimal capture time data to generate a capture execution data package containing three-dimensional spatial coordinate values and time point control information. The captured execution data packets are converted into data formats according to the communication protocol requirements and encapsulated to generate new capture and execution instructions that are adapted to the execution mechanism interface. A new grabbing and execution instruction is issued to the execution agency, driving the execution agency to perform grabbing actions according to the new time point and coordinate position; Throughout the entire process of issuing new grabbing and execution instructions and physical grabbing and execution, the original calculation data is maintained, and the triggering of instructions to re-acquire images and identify features in disordered incoming material areas is blocked.
9. The flexible structured smart automated production system of claim 1, wherein, The steps involved in iterating until all materials have been grabbed include: After completing the physical grabbing and detachment of the current first target material, the material is cleaned up and deleted from the data set to be grabbed. Update the current operational status of the executing agency to "awaiting readiness"; Extract the material data of the next sequential position from the records in the dataset to be crawled, and update it with the new subsequent material to be crawled; Repeatedly run disturbance prediction, trajectory envelope generation, spatiotemporal window calculation, and capture time series rearrangement operations; The physical motion data of the surrounding materials is updated in real time with each grabbing and detaching action, which serves as a constraint on the next material grabbing route until all materials recorded in the disordered material receiving area have been grabbed and transferred.
