Machine vision-based material grasping decision method and system

Abstract

The present application relates to the field of intelligent grabbing robots, in particular to a material grabbing decision method and system based on machine vision, which adopts an intelligent decision-making process based on machine vision, including environment perception and twin construction, single body segmentation and shape compensation, physical simulation parameter assignment, and grabbing risk assessment feedback, to build a grabbing decision implementation method for single bodies in a complex and chaotic material stacking scene. Compared with the existing nearby grabbing scheme, the method can judge the radiation impact generated during grabbing in the early stage, thereby deciding the optimal grabbing scheme, and effectively reducing the occurrence of multiple additional damage to goods due to material pile collapse and sliding during grabbing.

Description

Technical Field

[0001] This invention relates to the field of intelligent grasping robots, specifically to a material grasping decision-making method and system based on machine vision. Background Technology

[0002] In scenarios such as express delivery sorting and warehousing logistics, large quantities of materials such as cardboard boxes, parcels, and soft packaging are often stacked in an irregular and high-density manner. These stacked materials usually need to be picked up and sent to relevant equipment by humans or robots for sorting and diversion, so as to distribute the items to the next level of logistics points.

[0003] In existing technologies, whether manual or robotic, most grasping logic control methods employ the principles of "grabbing only what is visible," "grabbing the nearest priority," or "grabbing the highest point first." In particular, manual methods can only prioritize grasping from the lower middle part of the material pile closest to the operator (equipment). This causes the support between materials to be constantly broken during the grasping process, resulting in frequent partial collapses of the material pile and material rolling, leading to material damage and economic losses. Summary of the Invention

[0004] The purpose of this invention is to provide a machine vision-based material handling decision-making method and system to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Machine vision-based material handling decision-making methods include:

[0007] Real-time visual data from multiple perspectives of the material stacking scene is acquired in real time, and multiple sets of real-time visual data are spatially stacked to establish a three-dimensional point cloud model of the material stacking scene.

[0008] Based on the 3D point cloud model, the boundary identification and segmentation of material unit objects are performed. Based on the visible structure of the existing material unit objects, the invisible parts are structurally compensated to obtain the unit object model and update the 3D point cloud model.

[0009] The physical simulation parameters of the space where the three-dimensional point cloud model is located are assigned, including the gravity assignment acting on the environment and the collision volume and weight assignment acting on the individual object model.

[0010] Establish a physical collision model for the grasping part, and simulate the grasping and extraction of several individual objects to determine the falling situation of several surrounding individual objects. Assign influence values ​​based on the number of falling objects and the diffusion range.

[0011] Based on the influence assignment, several individual objects are sorted in ascending order to establish a grasping decision sequence, which is used to guide the grasping unit to grasp the individual objects of the stacked materials in sequence.

[0012] As a further aspect of the present invention: the process of establishing the crawling decision sequence includes the following steps:

[0013] The first grasping decision sequence is established by selecting a number of individual objects based on the first or first position of the ascending sequence and statistically analyzing the drop situation of their grasping and extraction simulation results.

[0014] The individual objects filtered by the first grasping decision sequence are sorted in ascending order, and the individual objects are reselected and the grasping extraction simulation results are statistically analyzed to establish a second grasping decision sequence.

[0015] After the first grasping decision sequence is executed, the 3D point cloud model is updated based on real-time visual data, the priority of the second grasping decision sequence is prioritized, and the grasping decision sequence is evaluated and updated cyclically.

[0016] As a further aspect of the present invention: when establishing the first grasping decision sequence and the second grasping decision sequence, the statistical analysis of the grasping and extraction simulation results also includes an extended statistical process for the redundant radiation range, which is used to characterize the outward radiation setting range of the edge of the fall situation based on the grasping and extraction simulation results.

[0017] The grasping unit also includes motion trajectory and attitude information, which are used for synchronization in the three-dimensional point cloud model to limit the simulated grasping angle of the grasping unit in the grasping and extraction simulation.

[0018] As a further aspect of the present invention: the step of assigning physical simulation parameters to the space where the three-dimensional point cloud model is located includes:

[0019] Information is identified based on visual data and matched with a material database to obtain corresponding physical data of the individual objects. The material database is used to record volume and weight information when individual materials are put into storage. The physical data of the individual objects includes volume structure data and weight data.

[0020] If the amount of identification information data cannot match the individual object, then the volume structure data of the material database is matched based on the structural information of the visible structure to obtain similar volume structure data and associated weight data, and then the individual object model is assigned a value.

[0021] As a further aspect of the present invention, it also includes the following steps:

[0022] Based on a pre-established database of commonly used materials for material packaging, the outer layer material of individual objects is matched;

[0023] The physical simulation parameters of the single object are assigned based on the matching results of the outer layer material. The matching results are used to characterize the surface tribophysical properties of the single object.

[0024] This invention aims to provide a machine vision-based material handling decision-making system, comprising:

[0025] The scene synchronization module is used to acquire real-time visual data of multiple perspectives of the material stacking scene in real time, and to spatially stack multiple sets of real-time visual data to establish a three-dimensional point cloud model of the material stacking scene.

[0026] The single-unit segmentation module is used to identify and segment the boundaries of material single-unit objects based on the 3D point cloud model. It performs structural compensation on the invisible parts based on the visible structure of the existing material single-unit objects to obtain the single-unit object model and update the 3D point cloud model.

[0027] The simulation assignment module is used to assign physical simulation parameters to the space where the three-dimensional point cloud model is located. The physical simulation parameters include the gravity assignment acting on the environment and the collision volume and weight assignment acting on the individual object model.

[0028] The grabbing and assignment module is used to establish the physical collision model of the grabbing part, and to simulate the grabbing and extraction of several individual objects, determine the falling situation of several surrounding individual objects, and assign influence values ​​based on the number of falls and the diffusion range.

[0029] The grasping decision module is used to sort several individual objects in ascending order based on influence assignment to establish a grasping decision sequence, which is used to guide the grasping unit to grasp the individual objects of the stacked materials in sequence.

[0030] As a further aspect of the present invention: the grasping decision module includes:

[0031] The first sequence establishment unit is used to select a number of individual objects based on the first or first position of the ascending sequence, and to statistically analyze the drop situation of the capture and extraction simulation results to establish the first capture decision sequence.

[0032] The second sequence establishment unit is used to sort the several individual objects after the first grasping decision sequence in ascending order, reselect the individual objects and perform statistical analysis of the grasping extraction simulation results to establish the second grasping decision sequence.

[0033] The sequence loop synchronization unit is used to update the 3D point cloud model based on real-time visual data after the first grasping decision sequence is executed, prioritize the second grasping decision sequence, and cyclically evaluate and update the grasping decision sequence.

[0034] As a further aspect of the present invention: when establishing the first grasping decision sequence and the second grasping decision sequence, the statistical analysis of the grasping and extraction simulation results also includes an extended statistical process for the redundant radiation range, which is used to characterize the outward radiation setting range of the edge of the fall situation based on the grasping and extraction simulation results.

[0035] The grasping unit also includes motion trajectory and attitude information, which are used for synchronization in the three-dimensional point cloud model to limit the simulated grasping angle of the grasping unit in the grasping and extraction simulation.

[0036] As a further embodiment of the present invention: the simulation assignment module includes:

[0037] The precise assignment unit is used to identify information of multiple individual material objects based on visual data and match them with the material database to obtain the corresponding individual physical data. The material database is used to record volume and weight information when individual material objects are put into storage. The individual physical data includes volume structure data and weight data.

[0038] The fuzzy assignment unit is used to perform volumetric structure data matching on the material database based on the visible structure information if the amount of identification information data cannot match the individual object, so as to obtain similar volumetric structure data and associated weight data, and assign values ​​to the individual object model.

[0039] As a further aspect of the present invention, it also includes:

[0040] The material matching unit is used to match the outer layer material of a single object based on a pre-established database of commonly used materials for material packaging.

[0041] The friction assignment unit is used to assign physical simulation parameters to a single object based on the matching results of the outer layer material. The matching results are used to characterize the surface tribophysical properties of the single object.

[0042] Compared with existing technologies, the beneficial effects of this invention are as follows: By adopting an intelligent decision-making process based on machine vision, which includes environmental perception and twin construction, single-unit segmentation and shape compensation, physical simulation parameter assignment, and grasping risk assessment feedback, a method for grasping single materials in complex and chaotic material stacking scenarios is established. Compared with existing nearby grasping schemes, it can judge the radiation impact generated by grasping in the early stage of grasping, thereby deciding to select the optimal grasping scheme, which can effectively reduce the occurrence of multiple additional damages to goods caused by the collapse and slippage of material stacks during grasping. Attached Figure Description

[0043] Figure 1 This is a flowchart of a machine vision-based material handling decision-making method.

[0044] Figure 2 This is a flowchart illustrating the establishment of the grasping decision sequence in a machine vision-based material grasping decision method.

[0045] Figure 3 This is a diagram showing the components of a machine vision-based material handling decision system.

[0046] Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0049] like Figure 1 As shown, a machine vision-based material handling decision-making method according to an embodiment of the present invention includes the following steps:

[0050] S10: Real-time visual data of multiple perspectives of the material stacking scene is acquired in real time, and multiple sets of real-time visual data are spatially stacked to establish a three-dimensional point cloud model of the material stacking scene.

[0051] S20: Based on the 3D point cloud model, the boundary identification and segmentation of material unit objects are performed. Based on the visible structure of the existing material unit objects, the invisible parts are structurally compensated to obtain the unit object model and update the 3D point cloud model.

[0052] S30, assign physical simulation parameters to the space where the three-dimensional point cloud model is located. The physical simulation parameters include the gravity assigned to the environment and the collision volume and weight assigned to the individual object model.

[0053] S40, establish a physical collision model of the grasping part, and simulate the grasping and extraction of several individual objects to determine the falling situation of several surrounding individual objects, and assign influence values ​​based on the number of falls and the diffusion range.

[0054] S50, based on the influence assignment, several individual objects are sorted in ascending order to establish a grasping decision sequence, which is used to guide the grasping unit to grasp the individual objects of the stacked materials in sequence.

[0055] This embodiment presents a machine vision-based material handling decision-making method applicable to situations involving large quantities of chaotic, high-density, irregularly stacked materials, such as in express delivery sorting and warehousing scenarios. By constructing a digital twin model of the stacked scenario and utilizing deep learning to segment and identify individual objects, and assigning physical characteristics, it simulates the collapse of the material pile during the handling process. This allows for the generation of handling decisions based on minimizing impact, reducing the probability and scope of material falls, collisions, and collapses during operations. In express delivery warehousing scenarios, when large quantities of goods enter the warehouse and are sorted, they are currently mostly picked up manually and sent to corresponding sorting equipment for classification. When using manual methods, picking usually starts from the edge of the material stack. Therefore, during this process, other items piled above will continuously fall and roll downwards during sorting, potentially causing damage to some items and resulting in transportation losses, leading to related damage losses and compensation issues. The solution in this embodiment is to use a surrounding setting... Multiple cameras acquire image data in real time, and twin modeling is performed based on this image data (high precision is not required, only the ability to effectively distinguish the structure and edges of the object is needed, so the use of radar is not necessary, but depth cameras and other solutions can be used to assist in the acquisition of depth information and achieve faster twin modeling). The structural boundary recognition of the twin 3D point cloud model can be performed to delineate and segment the boundaries of individual objects, and different structures can be split into different individual objects. Thus, the remaining parts can be structurally compensated based on the shape, size and structure that can be obtained from the outside, to obtain a possible complete shape and structure. At this time, the individual object includes the exposed part and the part that is piled up inside. By assigning physical parameters to the entire 3D point cloud model, the physical interference simulation during extraction can be performed, including the falling and rolling of the object above when it loses its support. Thus, the impact diffusion caused by the operation can be predicted in advance, and the basis for the grasping decision can be obtained to establish a grasping decision sequence with minimal impact.

[0056] like Figure 2 As shown, in another preferred embodiment of the present invention, the process of establishing the crawling decision sequence specifically includes:

[0057] S51, select a number of individual objects based on the first or first position of the ascending sequence, and statistically analyze the drop situation of the capture and extraction simulation results to establish the first capture decision sequence.

[0058] S52, sort the individual objects after filtering by the first grasping decision sequence in ascending order, reselect the individual objects and perform statistical analysis of the grasping extraction simulation results to establish a second grasping decision sequence.

[0059] S53, after the first grasping decision sequence is executed, the three-dimensional point cloud model is updated based on real-time visual data, the priority of the second grasping decision sequence is prioritized, and the grasping decision sequence is evaluated and updated cyclically.

[0060] In this embodiment, the establishment of the grasping decision sequence is supplemented because the grasping decision is based on the possible impact of grasping a specific single object. After a grasp, the material pile will change to some extent. Regardless of the extent of the impact after the grasping action, if only one grasping decision sequence is set, there will be a waiting gap in the continuous grasping. Therefore, N grasping decision sequences are set (N is greater than or equal to 2, and this is only explained when N=2). Based on the impact range after the actual execution of the first grasping decision sequence, the selection range of the second grasping decision sequence is updated and excluded, thereby establishing a dynamic decision loop. The "first or sequential fixed number of single objects" is also given. The number depends on the number of grasping units that can operate simultaneously.

[0061] As another preferred embodiment of the present invention, when establishing the first grasping decision sequence and the second grasping decision sequence, the statistical analysis of the grasping and extraction simulation results also includes an extended statistical process for the redundant radiation range. The redundant radiation range is used to characterize the outward radiation setting range of the edge of the fall situation based on the grasping and extraction simulation results.

[0062] The grasping unit also includes motion trajectory and attitude information, which are used for synchronization in the three-dimensional point cloud model to limit the simulated grasping angle of the grasping unit in the grasping and extraction simulation.

[0063] In this embodiment, an additional redundant radiation range setting is added. That is, when establishing the range affected by the first grasping decision sequence, an extension area is added outward based on the boundary of the influence range of the physical simulation. This is used to provide a certain allowable error range for the simulated influence range, so as to avoid the situation where the simulated range is smaller than the actual range, which would cause difficulties in the execution of the second grasping decision sequence.

[0064] In another preferred embodiment of the present invention, the step of assigning physical simulation parameters to the space where the three-dimensional point cloud model is located includes:

[0065] Information is identified based on visual data and matched with a material database to obtain corresponding physical data of the individual objects. The material database is used to record volume and weight information when individual materials are put into storage. The physical data of the individual objects includes volume structure data and weight data.

[0066] If the amount of identification information data cannot match the individual object, then the volume structure data of the material database is matched based on the structural information of the visible structure to obtain similar volume structure data and associated weight data, and then the individual object model is assigned a value.

[0067] In this embodiment, the steps for assigning physical simulation parameters are explained. Specifically, in this assignment of physical simulation parameters, the volumetric structure data is related to the structural compensation in step S20. Structural compensation is based on the known visible structure to compensate for the invisible parts, obtaining a basic single-object model. This single-object model is a collision-free entity, used only for basic model construction. However, physical simulation requires obtaining the actual collision volume of the single-object, which means that structural dimensions need to be assigned to establish a collision-capable entity model. Here, a material database is introduced, which is the warehousing record information of express delivery items. This information is obtained when the express delivery item is sent out. The data, including volume and weight information, has already been statistically obtained. Therefore, more reliable values ​​can be assigned based on this information, including two cases: accurate assignment and fuzzy assignment. Accurate assignment is used when there is visual data in the visible structural part of the individual object that can be used to identify the individual object's identification number. This allows for matching the exact corresponding physical data of the individual object in the material database. However, when limited information cannot accurately match the individual object, similarity matching can be performed between the volume and structural data in the material database and the individual object model. This allows for the selection of a more similar object as the result of fuzzy matching, making the simulation results after assignment closer to the correct situation.

[0068] As another preferred embodiment of the present invention, the method further includes the following steps:

[0069] Based on a pre-established database of commonly used materials for material packaging, the outer layer material of individual objects is matched;

[0070] The physical simulation parameters of the single object are assigned based on the matching results of the outer layer material. The matching results are used to characterize the surface tribophysical properties of the single object.

[0071] In this embodiment, the assignment of the frictional physical properties of the surface material is added to the physical simulation parameter assignment. By assigning this parameter, the situation of adjacent objects being carried outward can be more accurately determined during the grasping and extraction simulation, thereby achieving better simulation results.

[0072] like Figure 3 As shown, the present invention also provides a material handling decision system based on machine vision, which includes:

[0073] The scene synchronization module 100 is used to acquire real-time visual data of multiple perspectives of the material stacking scene in real time, and to spatially stack multiple sets of real-time visual data to establish a three-dimensional point cloud model of the material stacking scene.

[0074] The single-unit segmentation module 200 is used to identify and segment the boundaries of material single-unit objects based on the three-dimensional point cloud model, and to perform structural compensation on the invisible parts based on the visible structure of the existing material single-unit objects in order to obtain the single-unit object model and update the three-dimensional point cloud model.

[0075] The simulation assignment module 300 is used to assign physical simulation parameters to the space where the three-dimensional point cloud model is located. The physical simulation parameters include the gravity assignment acting on the environment and the collision volume and weight assignment acting on the individual object model.

[0076] The grasping and assignment module 400 is used to establish the physical collision model of the grasping part, and to perform grasping and extraction simulation on several individual objects, judge the falling situation of several surrounding individual objects, and assign influence values ​​based on the number of falls and the diffusion range.

[0077] The grasping decision module 500 is used to sort several individual objects in ascending order based on influence assignment to establish a grasping decision sequence, which is used to guide the grasping unit to grasp the individual objects of the stacked materials in sequence.

[0078] In another preferred embodiment of the present invention, the crawling decision module includes:

[0079] The first sequence establishment unit is used to select a number of individual objects based on the first or first position of the ascending sequence, and to statistically analyze the drop situation of the capture and extraction simulation results to establish the first capture decision sequence.

[0080] The second sequence establishment unit is used to sort the several individual objects after the first grasping decision sequence in ascending order, reselect the individual objects and perform statistical analysis of the grasping extraction simulation results to establish the second grasping decision sequence.

[0081] The sequence loop synchronization unit is used to update the 3D point cloud model based on real-time visual data after the first grasping decision sequence is executed, prioritize the second grasping decision sequence, and cyclically evaluate and update the grasping decision sequence.

[0082] As another preferred embodiment of the present invention, when establishing the first grasping decision sequence and the second grasping decision sequence, the statistical analysis of the grasping and extraction simulation results also includes an extended statistical process for the redundant radiation range. The redundant radiation range is used to characterize the outward radiation setting range of the edge of the fall situation based on the grasping and extraction simulation results.

[0083] The grasping unit also includes motion trajectory and attitude information, which are used for synchronization in the three-dimensional point cloud model to limit the simulated grasping angle of the grasping unit in the grasping and extraction simulation.

[0084] In another preferred embodiment of the present invention, the simulation assignment module includes:

[0085] The precise assignment unit is used to identify information of multiple individual material objects based on visual data and match them with the material database to obtain the corresponding individual physical data. The material database is used to record volume and weight information when individual material objects are put into storage. The individual physical data includes volume structure data and weight data.

[0086] The fuzzy assignment unit is used to perform volumetric structure data matching on the material database based on the visible structure information if the amount of identification information data cannot match the individual object, so as to obtain similar volumetric structure data and associated weight data, and assign values ​​to the individual object model.

[0087] As another preferred embodiment of the present invention, it further includes:

[0088] The material matching unit is used to match the outer layer material of a single object based on a pre-established database of commonly used materials for material packaging.

[0089] The friction assignment unit is used to assign physical simulation parameters to a single object based on the matching results of the outer layer material. The matching results are used to characterize the surface tribophysical properties of the single object.

[0090] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0091] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the disclosure in the specification and embodiments. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0092] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A material handling decision-making method based on machine vision, characterized in that, Include: Real-time visual data from multiple perspectives of the material stacking scene is acquired in real time, and multiple sets of real-time visual data are spatially stacked to establish a three-dimensional point cloud model of the material stacking scene. Based on the 3D point cloud model, the boundary identification and segmentation of material unit objects are performed. Based on the visible structure of the existing material unit objects, the invisible parts are structurally compensated to obtain the unit object model and update the 3D point cloud model. The physical simulation parameters of the space where the three-dimensional point cloud model is located are assigned, including the gravity assignment acting on the environment and the collision volume and weight assignment acting on the individual object model. Establish a physical collision model for the grasping part, and simulate the grasping and extraction of several individual objects to determine the falling situation of several surrounding individual objects. Assign influence values ​​based on the number of falling objects and the diffusion range. Based on the influence assignment, several individual objects are sorted in ascending order to establish a grasping decision sequence, which is used to guide the grasping unit to grasp the individual objects of the stacked materials in sequence.

2. The material handling decision-making method based on machine vision according to claim 1, characterized in that, The process of establishing the crawling decision sequence includes the following steps: The first grasping decision sequence is established by selecting a number of individual objects based on the first or first position of the ascending sequence and statistically analyzing the drop situation of their grasping and extraction simulation results. The individual objects filtered by the first grasping decision sequence are sorted in ascending order, and the individual objects are reselected and the grasping extraction simulation results are statistically analyzed to establish a second grasping decision sequence. After the first grasping decision sequence is executed, the 3D point cloud model is updated based on real-time visual data, the priority of the second grasping decision sequence is prioritized, and the grasping decision sequence is evaluated and updated cyclically.

3. The material handling decision-making method based on machine vision according to claim 2, characterized in that, When establishing the first grasping decision sequence and the second grasping decision sequence, the statistical analysis of the grasping and extraction simulation results also includes an extended statistical process for the redundant radiation range. The redundant radiation range is used to characterize the outward radiation setting range of the edge of the fall situation based on the grasping and extraction simulation results. The grasping unit also includes motion trajectory and attitude information, which are used for synchronization in the three-dimensional point cloud model to limit the simulated grasping angle of the grasping unit in the grasping and extraction simulation.

4. The material handling decision-making method based on machine vision according to claim 3, characterized in that, The step of assigning physical simulation parameters to the space where the 3D point cloud model is located includes: Information is identified based on visual data and matched with a material database to obtain corresponding physical data of the individual objects. The material database is used to record volume and weight information when individual materials are put into storage. The physical data of the individual objects includes volume structure data and weight data. If the amount of identification information data cannot match the individual object, then the volume structure data of the material database is matched based on the structural information of the visible structure to obtain similar volume structure data and associated weight data, and then the individual object model is assigned a value.

5. The material handling decision-making method based on machine vision according to claim 4, characterized in that, It also includes the following steps: Based on a pre-established database of commonly used materials for material packaging, the outer layer material of individual objects is matched; The physical simulation parameters of the single object are assigned based on the matching results of the outer layer material. The matching results are used to characterize the surface tribophysical properties of the single object.

6. A material handling decision-making system based on machine vision, characterized in that, Include: The scene synchronization module is used to acquire real-time visual data of multiple perspectives of the material stacking scene in real time, and to spatially stack multiple sets of real-time visual data to establish a three-dimensional point cloud model of the material stacking scene. The single-unit segmentation module is used to identify and segment the boundaries of material single-unit objects based on the 3D point cloud model. It performs structural compensation on the invisible parts based on the visible structure of the existing material single-unit objects to obtain the single-unit object model and update the 3D point cloud model. The simulation assignment module is used to assign physical simulation parameters to the space where the three-dimensional point cloud model is located. The physical simulation parameters include the gravity assignment acting on the environment and the collision volume and weight assignment acting on the individual object model. The grabbing and assignment module is used to establish the physical collision model of the grabbing part, and to simulate the grabbing and extraction of several individual objects, determine the falling situation of several surrounding individual objects, and assign influence values ​​based on the number of falls and the diffusion range. The grasping decision module is used to sort several individual objects in ascending order based on influence assignment to establish a grasping decision sequence, which is used to guide the grasping unit to grasp the individual objects of the stacked materials in sequence.

7. The material handling decision system based on machine vision according to claim 6, characterized in that, The crawling decision module includes: The first sequence establishment unit is used to select a number of individual objects based on the first or first position of the ascending sequence, and to statistically analyze the drop situation of the capture and extraction simulation results to establish the first capture decision sequence. The second sequence establishment unit is used to sort the several individual objects after the first grasping decision sequence in ascending order, reselect the individual objects and perform statistical analysis of the grasping extraction simulation results to establish the second grasping decision sequence. The sequence loop synchronization unit is used to update the 3D point cloud model based on real-time visual data after the first grasping decision sequence is executed, prioritize the second grasping decision sequence, and cyclically evaluate and update the grasping decision sequence.

8. The material handling decision system based on machine vision according to claim 7, characterized in that, When establishing the first grasping decision sequence and the second grasping decision sequence, the statistical analysis of the grasping and extraction simulation results also includes an extended statistical process for the redundant radiation range. The redundant radiation range is used to characterize the outward radiation setting range of the edge of the fall situation based on the grasping and extraction simulation results. The grasping unit also includes motion trajectory and attitude information, which are used for synchronization in the three-dimensional point cloud model to limit the simulated grasping angle of the grasping unit in the grasping and extraction simulation.

9. The machine vision-based material handling decision system according to claim 8, characterized in that, The simulation assignment module includes: The precise assignment unit is used to identify information of multiple individual material objects based on visual data and match them with the material database to obtain the corresponding individual physical data. The material database is used to record volume and weight information when individual material objects are put into storage. The individual physical data includes volume structure data and weight data. The fuzzy assignment unit is used to perform volumetric structure data matching on the material database based on the visible structure information if the amount of identification information data cannot match the individual object, so as to obtain similar volumetric structure data and associated weight data, and assign values ​​to the individual object model.

10. The machine vision-based material handling decision system according to claim 9, characterized in that, Also includes: The material matching unit is used to match the outer layer material of a single object based on a pre-established database of commonly used materials for material packaging. The friction assignment unit is used to assign physical simulation parameters to a single object based on the matching results of the outer layer material. The matching results are used to characterize the surface tribophysical properties of the single object.

Images