Product sorting and processing system and method based on three-dimensional vision

Abstract

The present invention discloses a product sorting and processing system and method based on three-dimensional vision. The system includes: a sorting controller; a sorting pool mechanism, including a sorting pool for placing materials; a three-dimensional vision mechanism, which is arranged above the sorting pool and connected to the sorting controller, and is used to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool under the control of the sorting controller, identify the identification and positioning information corresponding to the target material based on the three-dimensional image of the material, and send the identification and positioning information to the sorting controller; a sorting execution mechanism, which is arranged at the periphery of the sorting pool and connected to the sorting controller, and is used to sort materials according to the identification and positioning information under the control of the sorting controller. The system can ensure the accuracy and executability of the sorting process to avoid the inability to sort materials due to the shape of the target material, mutual occlusion or mutual stacking.

Description

Technical Field

[0001] The present invention relates to the field of machine vision control technology, and in particular to a product sorting and processing system and method based on three-dimensional vision. Background Art

[0002] In recent years, sorting technology has been increasingly used in warehousing, logistics and other industries, and the degree of automation of sorting has also become higher and higher. Existing sorting systems are mainly based on barcode recognition, QR code recognition or two-dimensional visual recognition technology. Among them, the scope of use of barcode recognition and QR code recognition is limited to the objects to be sorted that need to be marked with identification codes such as barcodes or QR codes. Two-dimensional visual recognition requires the objects to be sorted to be processed in advance so that there is no occlusion, stacking or other interference between the objects to be sorted; moreover, the two-dimensional visual recognition process can only provide coordinate information of the two-dimensional plane, and when sorting objects with special shapes, it is easy to cause the sorting process to fail. That is, the sorting system based on two-dimensional visual recognition is mainly used when the objects to be sorted have no special shapes and there is no occlusion or stacking between the objects to be sorted. Summary of the invention

[0003] Aiming at the problem that a sorting system based on two-dimensional visual recognition may affect the normal operation of the sorting work due to special shapes or mutual occlusion and stacking of sorted objects during the sorting process, a product sorting processing system and method based on three-dimensional vision are provided.

[0004] The present invention provides a product sorting and processing system based on three-dimensional vision, comprising:

[0005] Sorting controller;

[0006] A sorting pool mechanism, including a sorting pool for placing materials;

[0007] A three-dimensional vision mechanism is arranged above the sorting pool and connected to the sorting controller, and is used to collect a three-dimensional image of a material corresponding to the material placed on the sorting pool under the control of the sorting controller, identify identification and positioning information corresponding to the target material based on the three-dimensional image of the material, and send the identification and positioning information to the sorting controller;

[0008] A sorting execution mechanism is arranged at the periphery of the sorting pool and connected to the sorting controller, and is used for sorting materials according to the identification and positioning information under the control of the sorting controller.

[0009] Preferably, the sorting pool mechanism further comprises a sorting motion unit connected to the sorting pool and the sorting controller, and configured to flip the material placed on the sorting pool under the control of the sorting controller.

[0010] Preferably, the sorting pool comprises a sorting column and at least one sorting ring arranged on the periphery of the sorting column; the sorting motion unit is connected to the sorting column, and is used to control the sorting column to reciprocate along the axial direction of the sorting column; the sorting motion unit is connected to the sorting ring, and is used to control the sorting ring to reciprocate along the axial direction of the sorting ring.

[0011] Preferably, the three-dimensional vision mechanism includes a three-dimensional camera and a vision processing unit. The three-dimensional camera is arranged directly above the sorting pool and is used to collect a three-dimensional image of the material corresponding to the material placed on the sorting pool; the vision processing unit is connected to the three-dimensional camera and is used to identify the identification and positioning information corresponding to the target material based on the three-dimensional image of the material.

[0012] Preferably, the three-dimensional vision mechanism also includes a visual light source, which is arranged directly above the sorting pool and directly below the three-dimensional camera.

[0013] Preferably, the sorting execution mechanism includes at least one material receiving port, and the sorting controller is connected to at least one sorting robot arranged outside the sorting pool, sends material sorting instructions to the sorting robot according to the identification and positioning information, and receives the target material through the material receiving port.

[0014] The present invention provides a product sorting and processing method based on three-dimensional vision, comprising the following steps performed by a sorting controller:

[0015] Acquire sorting order information, wherein the sorting order information includes target materials;

[0016] Controlling the three-dimensional vision mechanism to collect a three-dimensional image of a material corresponding to the material placed on the sorting pool, and obtaining identification and positioning information corresponding to a target material identified based on the three-dimensional image of the material;

[0017] The sorting execution mechanism is controlled to sort materials according to the identification and positioning information.

[0018] Preferably, the sorting order information includes the quantity of materials corresponding to the target materials;

[0019] The controlling sorting execution mechanism performs material sorting according to the identification and positioning information, including:

[0020] According to the identification and positioning information corresponding to the target material, the sorting robot is controlled to pick up the target material from the sorting pool to the corresponding material receiving port;

[0021] Updating the current quantity corresponding to each of the target materials, and determining whether the current quantities corresponding to all of the target materials match the corresponding material quantities;

[0022] If the current quantities corresponding to all the target materials match the corresponding material quantities, the material sorting operation corresponding to the target product is completed.

[0023] Preferably, after updating the current quantity corresponding to each of the target materials and determining whether the current quantities corresponding to all the target materials match the corresponding material quantities, the product sorting processing method further includes:

[0024] If the current quantity corresponding to the target material does not match the corresponding material quantity, and there is no identification and positioning information corresponding to the unsorted target material, repeatedly controlling the three-dimensional vision mechanism to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool, and obtaining the identification and positioning information corresponding to the target material identified based on the three-dimensional image of the material; or

[0025] If the current quantity corresponding to the target material does not match the corresponding material quantity, and there is identification and positioning information corresponding to the unsorted target material, repeatedly execute the identification and positioning information corresponding to the target material to control the sorting robot to pick up the target material from the sorting pool to the corresponding material receiving port.

[0026] Preferably, the identification and positioning information includes position information and posture information;

[0027] According to the identification and positioning information corresponding to the target material, controlling the sorting robot to pick up the target material from the sorting pool to the corresponding material receiving port includes:

[0028] Determine a target robot according to the position information and a sorting area corresponding to each of the sorting robots;

[0029] According to the posture information, the picking angle of the target robot is adjusted to pick up the target material from the sorting pool to the corresponding material receiving port.

[0030] The above-mentioned product sorting and processing system and method based on three-dimensional vision collects the three-dimensional image of the material corresponding to the material placed on the sorting pool through the three-dimensional vision mechanism, and identifies the category, position information, posture information and other identification and positioning information corresponding to the target material based on the three-dimensional image of the material, which can help avoid the influence of the shape, mutual occlusion or stacking of the target material on the material identification and positioning, and can ensure the accuracy of the identification and positioning process; then guide the sorting execution mechanism to perform sequential and three-dimensional directional picking and sorting operations on the target material according to the identification and positioning information such as the category, position information and posture information of the target material, so as to ensure the accuracy and executability of the sorting process, and achieve the effect of flexible sorting of target materials of multiple categories, uncertain postures, occlusions and stacks, so as to avoid the inability to sort materials due to the shape, mutual occlusion or stacking of the target materials, and improve the efficiency and accuracy of material sorting. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic diagram of a product sorting and processing system based on three-dimensional vision in one embodiment of the present invention;

[0032] Figure 2 is a schematic diagram of a sorting pool mechanism in one embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of a three-dimensional vision mechanism in one embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of a sorting execution mechanism in one embodiment of the present invention;

[0035] Figure 5 is a schematic diagram of a material packaging mechanism in one embodiment of the present invention;

[0036] Figure 6 is a flow chart of a product sorting method based on three-dimensional vision in one embodiment of the present invention;

[0037] Figure 7 is another flow chart of a product sorting method based on three-dimensional vision in one embodiment of the present invention;

[0038] Figure 8 is another flow chart of a product sorting method based on three-dimensional vision in one embodiment of the present invention;

[0039] Fig. 9 It is a schematic diagram of the position information and posture information of the target material in one embodiment of the present invention.

[0040] Among them, 10, sorting controller; 20, sorting pool mechanism; 21, sorting pool; 211, sorting column; 212, sorting ring; 30, three-dimensional vision mechanism; 31, three-dimensional camera; 32, visual processing unit; 33, visual light source; 40, sorting execution mechanism; 41, sorting robot; 42, material receiving port; 50, manufacturing execution system; 60, material warehouse control mechanism; 70, material packaging mechanism; 71, material collection unit; 711, material collection channel; 72, plastic packaging machine; 721, first sensor; 73, inkjet printer; 74, material conveying unit. DETAILED DESCRIPTION

[0041] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] In the description of the present invention, it should be understood that the terms "longitudinal", "radial", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0043] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0044] In one embodiment, if Figure 1 As shown, a product sorting and processing system based on three-dimensional vision is provided. The three-dimensional vision-based system includes a sorting controller 10, a sorting pool mechanism 20 for placing materials, a three-dimensional vision mechanism 30 for performing three-dimensional vision recognition, and a sorting execution mechanism 40 for executing material sorting. The sorting controller 10 controls the three-dimensional vision mechanism 30 and the sorting execution mechanism 40 to work, so that the materials placed on the sorting pool 21 can be sorted without being restricted by the shape of the materials, whether there is mutual occlusion and stacking, so as to ensure the normal progress of the material sorting work.

[0045] The sorting controller 10 is a mechanism for realizing material sorting control, and can be specifically a control unit for executing a specific computer program to realize control of the sorting pool mechanism 20, the three-dimensional vision mechanism 30 and the sorting execution mechanism 40, so that the sorting pool mechanism 20, the three-dimensional vision mechanism 30 and the sorting execution mechanism 40 cooperate to complete the material sorting work.

[0046] The sorting pool mechanism 20 includes a sorting pool 21 for placing materials. The sorting pool 21 is a pre-set platform for placing materials so that the three-dimensional vision mechanism 30 can capture the three-dimensional image of the materials. For example, the sorting pool 21 can be a circular tabletop, and all materials that need to be sorted can be placed on the sorting pool 21 so that the three-dimensional vision mechanism 30 can capture the three-dimensional image of the materials and the sorting execution mechanism 40 can sort the materials. At this time, the materials placed on the sorting pool 21 are sorted objects.

[0047] The three-dimensional vision mechanism 30 is arranged above the sorting pool 21 and connected to the sorting controller 10, and is used to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool 21 under the control of the sorting controller 10, identify the identification and positioning information corresponding to the target material based on the three-dimensional image of the material, and send the identification and positioning information to the sorting controller 10. Among them, the three-dimensional image of the material is a three-dimensional image of the material placed on the sorting pool 21 captured by the three-dimensional vision mechanism 30. The identification and positioning information is information related to the target material based on the three-dimensional image of the material, including but not limited to the category, position information and posture information of the target material. It can be understood that the identification and positioning information such as the category, position information and posture information corresponding to the target material that can be identified through the three-dimensional image of the material captured by the three-dimensional vision mechanism 30 can help avoid the influence of the shape, mutual occlusion or mutual stacking of the target material on the material identification and positioning, so as to ensure the accuracy of the identification and positioning process. The three-dimensional vision mechanism 30 sends the identification and positioning information to the sorting controller 10, so that the sorting controller 10 controls the sorting execution mechanism 40 to sort the materials based on the identification and positioning information such as the category, position information and posture information corresponding to the target material, and can perform sorting and picking operations in an orderly and three-dimensional direction to ensure the accuracy and executability of the sorting process, so as to achieve the effect of flexible sorting of target materials of various types, uncertain postures, occlusions and stacks. The target material refers to the material that needs to be sorted out for assembling the target product. In this embodiment, the three-dimensional vision mechanism 30 is arranged above the sorting pool 21, which is conducive to the three-dimensional vision mechanism 30 to capture more accurate three-dimensional images of the material, which helps to improve the accuracy of material sorting.

[0048] The sorting execution mechanism 40 is arranged at the periphery of the sorting pool 21 and connected to the sorting controller 10, and is used to sort materials according to the identification and positioning information under the control of the sorting controller 10. The sorting execution mechanism 40 is a mechanism for performing material sorting. The sorting execution mechanism 40 is arranged at the periphery of the sorting pool 21 and connected to the sorting controller 10, and can perform sequential and three-dimensional picking and sorting operations according to the identification and positioning information such as the category, position information and posture information of the target material identified by the three-dimensional vision mechanism 30 obtained by the sorting controller 10, so as to achieve the effect of flexible sorting of target materials of multiple categories, uncertain postures, obstructions and stacking.

[0049] Before sorting begins, it is necessary to provide the sorting controller 10 with sorting order information corresponding to the target product and place the materials to be sorted on the sorting pool 21. The sorting order information includes at least one target material and the quantity of materials corresponding to each target material. Each target material corresponds to an original three-dimensional image, so that material identification can be performed based on the original three-dimensional image. During the sorting process, the sorting controller 10 first controls the three-dimensional vision mechanism 30 to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool 21, and performs material identification and positioning processing based on the three-dimensional image of the material and the original three-dimensional image corresponding to the target material in the sorting order information, thereby obtaining the identification and positioning information such as the category, position information and posture information corresponding to each target material in the three-dimensional image of the material, and sending the identification and positioning information to the sorting controller 10. After receiving the identification and positioning information, the sorting controller 10 controls the sorting actuator 40 to perform sequential, three-dimensional material picking and sorting operations based on the identification and positioning information, and picks out at least one target material corresponding to the sorting order information in turn until the quantity of each target material sorted out matches the material quantity, so as to achieve the effect of flexible sorting of target materials of multiple types, uncertain postures, obstructions and stacking.

[0050] It can be understood that after picking up all target materials identified by a three-dimensional image of a material, if the sorting of all target materials in the sorting order information is not completed, that is, if the current quantity of at least one target material does not match the material quantity, the three-dimensional vision mechanism 30 can be controlled to take the three-dimensional image of the material again and perform identification and positioning to re-acquire the identification and positioning information, and control the sorting execution mechanism 40 to perform material sorting operations based on the re-acquired identification and positioning information until all target materials are picked up according to their material quantities. Among them, the current quantity of the target material refers to the quantity of a certain category of target materials that have been picked up and sorted. That is, the sorting controller 10 can control the three-dimensional vision mechanism 30 to collect the three-dimensional image of the material layer by layer, identify, segment and locate the material in the three-dimensional image of the material, so as to obtain the identification and positioning information such as the category, position information and posture information corresponding to the target material, thereby guiding the sorting execution mechanism 40 to perform sequential and three-dimensional material picking and sorting operations based on the identification and positioning information, so as to achieve the purpose of flexible sorting of target materials of various categories, uncertain postures, obscured or stacked on each other, thereby effectively improving the material sorting efficiency and accuracy. Among them, flexible sorting refers to the adaptive response and sorting process to an uncertain environment, and flexibility here refers to a resilience to environmental changes or the uncertainty caused by environmental changes.

[0051] In the product sorting and processing system based on three-dimensional vision provided in the present embodiment, the three-dimensional image of the material corresponding to the material placed on the sorting pool 21 is collected by the three-dimensional vision mechanism 30, and the category, position information, posture information and other identification and positioning information corresponding to the target material are identified based on the three-dimensional image of the material, which can help avoid the influence of the shape, mutual occlusion or mutual stacking of the target material on the material identification and positioning, and can ensure the accuracy of the identification and positioning process; then guide the sorting execution mechanism 40 to perform sequential and three-dimensional picking and sorting operations on the target material according to the identification and positioning information such as the category, position information and posture information of the target material, so as to ensure the accuracy and executability of the sorting process, and achieve the effect of flexible sorting of target materials of multiple categories, uncertain postures, occlusions and stacks, so as to avoid the inability to sort materials due to the shape, mutual occlusion or mutual stacking of the target materials, and improve the efficiency and accuracy of material sorting.

[0052] In one embodiment, the sorting pool mechanism 20 further includes a sorting motion unit (not shown) connected to the sorting pool 21 and the sorting controller 10, which is used to flip the materials placed on the sorting pool 21 under the control of the sorting controller 10. Since the materials placed on the sorting pool 21 may block or stack each other, in order to reduce the impact of such blocking or stacking on material identification and positioning, the materials on the sorting pool 21 can be flipped to reduce the blocking or stacking of the materials, so that the three-dimensional image of the materials collected by the three-dimensional vision mechanism 30 can identify more target materials, thereby improving the efficiency and feasibility of material sorting. In order to improve the degree of automation of the product sorting and processing system and reduce the labor cost in the material sorting process, the sorting pool mechanism 20 may also include a sorting motion unit connected to the sorting pool 21 and the sorting controller 10 for flipping the material, so that the sorting motion unit can control the movement of the sorting pool 21 under the control of the sorting controller 10 to realize automatic flipping of the materials on the sorting pool 21, improve the material flipping efficiency, and reduce the occurrence of materials blocking or stacking each other, which helps to improve the material sorting efficiency.

[0053] In one embodiment, if Figure 2 As shown, the sorting pool 21 includes a sorting column 211 and at least one sorting ring 212 arranged on the periphery of the sorting column 211; the sorting motion unit is connected to the sorting column 211, and is used to control the sorting column 211 to reciprocate along the axial direction of the sorting column 211; the sorting motion unit is connected to the sorting ring 212, and is used to control the sorting ring 212 to reciprocate along the axial direction of the sorting ring 212. Among them, the sorting column 211 is a cylindrical part in the center part of the sorting pool 21, and the sorting ring 212 is an annular part arranged on the periphery of the sorting column 211. In order to prevent materials from falling between the sorting column 211 and the sorting ring 212 or between two adjacent sorting rings 212, it is necessary to ensure that the sorting column 211 and the sorting ring 212 or between two adjacent sorting rings 212 are tightly matched to make the gap smaller. It can be understood that the sorting motion unit is connected to the sorting column 211 and the sorting ring 212, so that the sorting motion unit can be used alone to control the sorting column 211 to reciprocate along the axial direction of the sorting column 211, and can also control the sorting ring 212 to reciprocate along the axial direction of the sorting ring 212 alone, or synchronously control the sorting column 211 to reciprocate along the axial direction of the sorting column 211 and control the sorting ring 212 to reciprocate along the axial direction of the sorting ring 212, so that when the sorting motion unit controls the sorting column 211 and / or the sorting ring 212 to reciprocate along their respective axial directions, the materials placed on the sorting column 211 and / or the sorting ring 212 move accordingly, so as to achieve the purpose of flipping the materials and avoid the materials from blocking or stacking each other to affect the normal material sorting work.

[0054] As an example, Figure 2As shown, the sorting column 211 is specifically a cylinder, and the end surface of the sorting column 211 for placing materials is a spherical surface. When the sorting motion unit drives the materials placed on the sorting column 211 to reciprocate along the axial direction of the sorting column 211, the materials are arranged in the direction of the sorting ring 212 to achieve the purpose of material flipping. The sorting ring 212 is specifically a circular cylinder, that is, the cross section of the sorting ring 212 is a circular ring, and the height of the inner ring of the circular cylinder is less than the height of the outer ring, so that the end surface of the sorting ring 212 for placing materials is an inclined surface. When the sorting motion unit drives the materials placed on the sorting ring 212 to reciprocate along the axial direction of the sorting ring 212, the materials are arranged in the direction of the sorting column 211 to achieve the purpose of material flipping. It can be understood that the sorting controller 10 can control the formation of different height differences between the sorting column 211 and the sorting ring 212 through the sorting motion unit to effectively organize the materials in the sorting pool 21 to avoid the situation where the materials are blocked and stacked on each other and affect the material sorting process.

[0055] In one embodiment, if Figure 3 As shown, the three-dimensional vision mechanism 30 includes a three-dimensional camera 31 and a vision processing unit 32. The three-dimensional camera 31 is arranged directly above the sorting pool 21, and is used to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool 21; the vision processing unit 32 is connected to the three-dimensional camera 31, and is used to identify the identification and positioning information corresponding to the target material based on the three-dimensional image of the material.

[0056] The three-dimensional camera 31 is a camera that can take three-dimensional images, and can be, but not limited to, the RGB-D three-dimensional camera 31 in this embodiment. In this embodiment, the three-dimensional camera 31 is arranged directly above the sorting pool 21, which can help prevent the three-dimensional image of the material taken from affecting its recognition and positioning effect due to a large angle deviation, thereby improving the accuracy of recognition and positioning and ensuring the accuracy of material sorting.

[0057] The visual processing unit 32 is a processing unit connected to the three-dimensional camera 31 for image recognition and positioning processing, and can be a computer device that can execute a specific algorithm for image recognition and positioning. The specific algorithm here can be a feature correlation recognition and positioning algorithm, or other existing algorithms that can achieve image recognition and positioning. It can be understood that the visual processing unit 32 can be independently set up with the sorting controller 10, or it can be integrated in the sorting controller 10 and integrated with the sorting controller 10 to simplify the structure of the product sorting processing system.

[0058] like Fig. 9As shown, a three-dimensional space coordinate system shared by the pre-constructed three-dimensional vision mechanism 30 and the sorting execution mechanism 40 is established. In the vision processing unit 32, a feature correlation recognition and positioning algorithm or other recognition and positioning algorithms are used to match the three-dimensional image of the material with the original three-dimensional image corresponding to the target material in the pre-received sorting order information, and recognition and positioning information including the category, position information (x, y, z), and attitude information (α, θ, φ) corresponding to the target material is obtained. Here, the category can be used to determine which type of target material a certain material in the three-dimensional image of the material specifically is in the sorting order information. The position information (x, y, z) here refers to the coordinates of the center point Q of the target material in the three-dimensional space coordinate system. The attitude information (α, θ, φ) here refers to the angle between the pre-set material direction of the target material and a certain coordinate axis in the three-dimensional space coordinate system. For example, the angle between the length direction of a cuboid target material and the X-axis in the three-dimensional space coordinate system is α in the attitude information. It can be understood that after receiving the recognition and positioning information, when the sorting execution mechanism 40 processes the target material, it can perform an orderly material picking and sorting operation according to the category in the recognition and positioning information, and adjust the picking position point and picking attitude according to the position information (x, y, z) and attitude information (α, θ, φ) in the recognition and positioning information, so as to realize the material picking and sorting operation in three-dimensional directions, achieve the effect of flexible sorting of target materials with multiple types, indefinite postures, occlusion, and stacking, and ensure the accuracy of material sorting.

[0059] In one embodiment, as Figure 3 shown, the three-dimensional vision mechanism 30 further includes a vision light source 33, and the vision light source 33 is arranged directly above the sorting pool 21 and directly below the three-dimensional camera 31. It can be understood that by arranging the vision light source 33 at the position directly above the sorting pool 21 and directly below the three-dimensional camera 31, it is used to provide a stable lighting environment during the process of the three-dimensional camera 31 taking the three-dimensional image of the material, so as to ensure the clarity of the three-dimensional image of the material collected by the three-dimensional camera 31, thereby helping to improve the accuracy of subsequent recognition and positioning.

[0060] Furthermore, the vision light source 33 is arranged in a ring between the sorting pool 21 and the three-dimensional camera 31 to provide a more stable lighting environment for the three-dimensional camera 31, and to avoid the influence of the lighting angle of the vision light source 33 on the material recognition and positioning effect of the materials at different positions on the sorting pool 21.

[0061] In one embodiment, as Figure 4As shown, the sorting execution mechanism 40 includes at least one receiving port 42, and the sorting controller 10 is connected to at least one sorting robot 41 arranged outside the sorting pool 21, sends a material sorting instruction to the sorting robot 41 according to the identification and positioning information, and receives the target material through the receiving port 42. Among them, the material sorting instruction is an instruction triggered by the sorting controller 10 to control the sorting robot 41 to perform material picking and sorting operations. At least one sorting robot 41 for picking up materials is arranged outside the sorting pool 21. Each sorting robot 41 can sort materials according to the identification and positioning information identified by the three-dimensional vision mechanism 30 under the control of the sorting controller 10, so as to pick up the target material to the corresponding receiving port 42, so as to achieve the purpose of material sorting.

[0062] As an example, after receiving the identification and positioning information including the category, location information and posture information of the target material, the sorting controller 10 forms a material sorting instruction for a certain target material, and sends the material sorting instruction to a sorting robot 41, so that the sorting robot 41 executes the material sorting instruction to pick up the corresponding target material to the receiving port 42. For example, the sorting controller 10 can sequentially form a material sorting instruction according to the category order of the target materials or the order of distance from the sorting robot 41, so as to realize a sequential material picking and sorting operation based on the sequentially formed material sorting instructions; send a material sorting instruction to the sorting robot 41 closest to the target material, so that the sorting robot 41 adjusts the picking angle based on the posture information of the target material, so as to realize a material picking operation with a three-dimensional direction, thereby achieving the effect of flexible sorting of target materials of multiple categories, uncertain postures, obstructions and stacking, so as to ensure the accuracy of material sorting.

[0063] As an example, when there are at least one sorting robot 41 and one receiving port 42, the material sorting purpose can be achieved at the lowest cost. As another example, when there are at least two sorting robots 41 and at least two receiving ports 42, each sorting robot 41 can place the picked-up target material on the receiving port 42 closest to the sorting robot 41 to improve the material sorting efficiency.

[0064] For example, when there are at least two sorting robots 41, each sorting robot 41 can be pre-configured with a sorting area in a pre-constructed three-dimensional space coordinate system shared by the three-dimensional vision mechanism 30 and the sorting execution mechanism 40; after the sorting robot 41 obtains the corresponding identification and positioning information, the position information (x, y, z) in the identification and positioning information is matched with the sorting area corresponding to each sorting robot 41 to determine the sorting robot 41 corresponding to the target material; the sorting robot 41 can pick up the target material in its sorting area to the corresponding material receiving port 42 according to the target material and material quantity in the sorting order information to complete the sorting operation of the target material, which helps to improve the material sorting efficiency.

[0065] Furthermore, the product sorting and processing system based on three-dimensional vision also includes a manufacturing execution system 50 connected to the sorting controller 10 for issuing sorting order information. The manufacturing execution system 50 (Manufacturing Execution System, referred to as MES) is a production information management system for the workshop execution layer of manufacturing enterprises. MES can provide enterprises with management modules including manufacturing data management, planning and scheduling management, production scheduling management, inventory management, quality management, human resource management, work center / equipment management, tool and tooling management, procurement management, cost management, project kanban management, production process control, bottom-level data integration analysis, and upper-level data integration decomposition, so as to create a solid, reliable, comprehensive and feasible manufacturing collaborative management platform for enterprises. It can be understood that the user can send sorting order information to the sorting controller 10 through MES, so that the sorting controller 10 controls the sorting motion unit in the sorting pool mechanism 20 to flip the material placed on the sorting pool 21 according to the sorting order information; and controls the three-dimensional vision mechanism 30 to adopt the three-dimensional image of the material and perform identification and positioning to obtain identification and positioning information; and then controls the sorting execution mechanism 40 to perform material sorting according to the identification and positioning information to achieve the purpose of material sorting.

[0066] In one embodiment, the product sorting and processing system based on three-dimensional vision also includes a material warehouse control mechanism 60 connected to the manufacturing execution system 50 and the sorting pool mechanism 20, and the material warehouse control mechanism 60 is used to transport materials to the sorting pool mechanism 20 according to the sorting order information. It can be understood that the material warehouse control mechanism 60 is a mechanism that stores materials in advance and can transport materials to the sorting pool mechanism 20 according to the sorting order information issued by the MES, so that the materials that need to be sorted are placed on the sorting pool 21. Specifically, the material warehouse control mechanism 60 is connected to the sorting pool mechanism 20 through a conveyor belt or other conveying mechanism to realize the conveyance of materials to the sorting pool 21, so as to avoid the cost of manual material handling and improve the efficiency of material delivery.

[0067] In one embodiment, the product sorting and processing system based on three-dimensional vision further includes a material packaging mechanism 70 connected to the sorting controller 10, which is used to package the sorted target materials. It can be understood that after the materials are sorted based on the sorting order information, the material packaging mechanism 70 can be used to package the sorted target materials to achieve integrated sorting and packaging processing, so as to improve the sorting and packaging efficiency of the target products and save the sorting and packaging costs.

[0068] In one embodiment, if Figure 5 As shown, the material packaging mechanism 70 includes a material collection unit 71 for collecting target materials, a plastic packaging machine 72 for packaging and packaging the target materials, and an inkjet printer 73 for inkjet coding, which are connected in sequence through a material conveying unit 74. The material collection unit 71 is connected to the sorting actuator 40 through a material collection channel 711. Among them, the material conveying unit 74 is a unit for realizing the transmission of materials between different positions, and the material conveying unit 74 includes but is not limited to the conveyor belt unit mentioned in the embodiment. Among them, the material collection unit 71 is a unit for collecting target materials on the receiving port 42 of the sorting actuator 40, which can be understood as a container for collecting and accommodating all target materials. The material collection channel 711 is a channel for connecting at least one receiving port 42 with the material collection unit 71, and the material collection channel 711 can be a transmission channel that can realize automatic transmission of target materials.

[0069] In this embodiment, after the sorting execution mechanism 40 sorts all the target materials in the sorting order information according to their material quantities, all the target materials in the material receiving port 42 can be collected into the material collecting unit 71 through the material collecting channel 711, and then the plastic packaging machine 72 is used to pack and package all the target materials collected in the material collecting unit 71 to package all the target materials in the packaging bag, and then the inkjet printer 73 is used to spray the packaging bag to complete the integrated processing of sorting, packaging and coding of the target products, thereby improving the processing efficiency.

[0070] Furthermore, the material packaging mechanism 70 further includes a first sensor 721 disposed at the entrance of the plastic packaging machine 72, which is used to trigger a packing and packaging instruction to the plastic packaging machine 72 when sensing the target material, so that the plastic packaging machine 72 packs and packages the target material. The first sensor 721 is disposed at the entrance of the plastic packaging machine 72, and is used to trigger a packing and packaging instruction to the plastic packaging machine 72 when sensing that the material collection unit 71 conveys the target material to the entrance of the plastic packaging machine 72, so that the plastic packaging machine 72 packs and packages the target material, thereby improving the automation of the material packaging mechanism 70 and effectively improving the processing efficiency.

[0071] Correspondingly, the material packaging mechanism 70 also includes a second sensor (not shown in the figure) arranged at the entrance of the inkjet printer 73, which is used to trigger a coding control instruction to the inkjet printer 73 when the target material is sensed, so that the inkjet printer 73 can code the packaging bag of the target material according to the coding control instruction, so as to improve the degree of automation of the material packaging mechanism 70 and effectively improve the processing efficiency.

[0072] In one embodiment, if Figure 6 As shown, a product sorting method based on three-dimensional vision is provided, and the product sorting method includes the following steps performed by the sorting controller 10:

[0073] S601: Acquire sorting order information, where the sorting order information includes target materials.

[0074] Among them, the sorting order information is used to reflect the images and quantity of all materials corresponding to a certain target product. The target material refers to the material that needs to be sorted out for assembling the target product. The sorting order information includes not only the target material, but also the material quantity corresponding to the target material and the original three-dimensional image. The material quantity refers to the quantity of the target material required to assemble the target product. The original three-dimensional image refers to the three-dimensional image corresponding to the pre-set target material, and subsequent three-dimensional visual recognition can be performed based on the original three-dimensional image.

[0075] Specifically, the sorting controller 10 can receive sorting order information corresponding to K target products, where the number of target products is at least one, that is, K is a positive integer, to achieve unified sorting of batch target products. The sorting order information corresponding to each target product includes N target materials, and the number of materials for each target material is M, where M and N are both positive integers. For example, the sorting controller 10 can receive sorting order information for 10 building block robots to be sorted at one time, and each building block robot must include target materials A, B and C, and the number of materials corresponding to target materials A, B and C are 6, 8 and 12 respectively.

[0076] In one embodiment, the sorting controller 10 can receive sorting order information directly input by the user through a client (such as a client that can implement a corresponding control program on a computer) so as to perform material sorting processing based on the sorting order information.

[0077] In another embodiment, the sorting controller 10 can receive sorting order information issued by the manufacturing execution system 50, that is, the sorting order information is based on the order information for sorting target products that is pre-uniformly managed and generated by the MES, so as to perform unified material sorting processing based on the sorting order information.

[0078] S602: Control the three-dimensional vision mechanism to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool, and obtain the identification and positioning information corresponding to the target material identified based on the three-dimensional image of the material.

[0079] In this embodiment, a three-dimensional space coordinate system shared by the three-dimensional vision mechanism 30 and the sorting execution mechanism 40 is pre-constructed as follows: Fig. 9 As shown; after the sorting controller 10 receives the sorting order information, the visual processing unit 32 that controls the three-dimensional visual mechanism 30 uses a feature correlation recognition and positioning algorithm or other recognition and positioning algorithm to match the three-dimensional image of the material with the original three-dimensional image corresponding to the target material in the sorting order information, and obtains the recognition and positioning information including the category, position information (x, y, z) and posture information (α, θ, φ) corresponding to the target material. The category here is used to determine which category of target material a certain material in the three-dimensional image of the material is in the sorting order information. The position information (x, y, z) here refers to the coordinates of the center point Q of the target material in the three-dimensional space coordinate system. The posture information (α, θ, φ) here refers to the angle between the pre-set material direction of the target material and a certain coordinate axis in the three-dimensional space coordinate system, such as the angle between the length direction of the target material of the cuboid and the X-axis in the three-dimensional space coordinate system is α in the posture information.

[0080] Since the three-dimensional vision mechanism 30 collects the three-dimensional image of the material corresponding to the material placed on the sorting pool 21, and uses the three-dimensional image of the material to match the original three-dimensional image to realize the recognition and positioning of the target material, the obtained recognition and positioning information includes information such as the category, position information and posture information of the target material, so that based on the recognition and positioning information such as the category, position information and posture information of the target material, the target material is picked and sorted in an orderly and three-dimensional direction, so as to achieve the effect of flexible sorting of target materials of multiple categories, uncertain postures, occluded and stacked, which can help avoid the influence of the shape of the target material, mutual occlusion or mutual stacking on the material recognition and positioning, and can ensure the accuracy of the recognition and positioning process.

[0081] S603: Control the sorting execution mechanism to sort materials according to the identification and positioning information.

[0082] Specifically, after the sorting controller 10 receives the identification and positioning information from the three-dimensional vision mechanism 30, since the three-dimensional vision mechanism 30 and the sorting execution mechanism 40 share the same three-dimensional space coordinate system, the sorting controller 10 controls the sorting execution mechanism 40 to perform sequential, three-dimensional material picking and sorting operations based on the received identification and positioning information containing the category, position information and posture information of the target material, so as to achieve the effect of flexible sorting of target materials of multiple categories, uncertain postures, occluded and stacked materials, so as to avoid the inability to sort materials due to the shape of the target materials, mutual occlusion or stacking, thereby improving the efficiency and accuracy of material sorting.

[0083] As an example, the sorting controller 10 can control the sorting execution mechanism 40 to perform sequential and three-dimensional material picking and sorting operations on all target materials identified in the identification and positioning information based on the category, position information, posture information and other identification and positioning information of the target material identified by the three-dimensional image of the material collected by the three-dimensional vision mechanism 30 each time, and update the material sorting results in real time; if the material sorting result is that the sorting is completed, the sorted target material can be transported to the next sorting processing link for processing, that is, transported to the material packaging mechanism 70 for material packaging; if the material sorting result is that the sorting is not completed, the three-dimensional vision mechanism 30 can be controlled to capture the three-dimensional image of the material again and perform identification and positioning, and the sorting execution mechanism 40 can be controlled to perform material sorting operations until all target materials are picked up according to their material quantities. That is, the sorting controller 10 can control the three-dimensional vision mechanism 30 to collect the three-dimensional image of the material layer by layer, identify, segment and locate the material in the three-dimensional image of the material, so as to obtain the identification and positioning information such as the category, position information and posture information corresponding to all target materials, so as to guide the sorting execution mechanism 40 to perform sequential and three-dimensional directional material picking and sorting operations based on the identification and positioning information, so as to achieve the purpose of flexible sorting of target materials of multiple categories, uncertain postures, obscured or stacked on each other, thereby effectively improving the material sorting efficiency and accuracy.

[0084] In the product sorting and processing method based on three-dimensional vision provided in the present embodiment, the three-dimensional image of the material corresponding to the material placed on the sorting pool 21 is collected by the three-dimensional vision mechanism 30, and the category, position information, posture information and other identification and positioning information corresponding to the target material are identified based on the three-dimensional image of the material; it can help to avoid the influence of the shape, mutual occlusion or stacking of the target material on the material identification and positioning, and ensure the accuracy of the identification and positioning process; then guide the sorting execution mechanism 40 to perform sequential and three-dimensional picking and sorting operations on the target material according to the identification and positioning information such as the category, position information and posture information of the target material, so as to achieve the effect of flexible sorting of target materials of multiple categories, uncertain postures, occlusions and stacks, so as to ensure the accuracy and executability of the sorting process, so as to avoid the inability to sort materials due to the shape, mutual occlusion or stacking of the target materials, and improve the efficiency and accuracy of material sorting.

[0085] In one embodiment, if Figure 7 As shown, the sorting order information includes the material quantity corresponding to the target material, then step S603, i.e. controlling the sorting execution mechanism 40 to sort the material according to the identification and positioning information, specifically includes the following steps:

[0086] S701: According to the identification and positioning information corresponding to the target material, the sorting robot is controlled to pick up the target material from the sorting pool to the corresponding receiving port.

[0087] Specifically, the sorting controller 10 can determine the sorting robot 41 that can sort the target material according to the identification and positioning information corresponding to the target material identified by the three-dimensional image of the material; and then control the sorting robot 41 to pick up the target material from the sorting pool 21 to the receiving port 42 corresponding to the sorting robot 41 according to a specific sorting order, so as to sort the target materials in sequence, that is, to perform sequential picking and sorting operations on the target materials. Since the identification and positioning information is obtained by identification and positioning processing based on the three-dimensional image of the material collected by the three-dimensional vision mechanism 30, compared with the two-dimensional vision technology, its identification and positioning results are more accurate and are not easily affected by the shape, posture, mutual occlusion or mutual stacking of the target material, which helps to ensure the smooth progress of the sorting work.

[0088] S702: Update the current quantity corresponding to each target material, and determine whether the current quantities corresponding to all target materials match the corresponding material quantities.

[0089] The current quantity of target materials refers to the quantity of target materials of a certain category that have been picked up and sorted out, specifically, the quantity of target materials of each category that have been sorted to the corresponding receiving port 42. Specifically, after receiving the sorting order information, the sorting controller 10 can create a counter for each target material, which is used to calculate the current quantity corresponding to each target material in real time, that is, when each target material is sorted to the corresponding receiving port 42 by the sorting robot 41, the current quantity corresponding to the target material is increased by 1, and it is determined whether the current quantity after the increase by 1 reaches the material quantity corresponding to the target material in the sorting order information, that is, it is determined one by one whether the current quantity corresponding to each target material matches the corresponding material quantity, so as to determine whether the corresponding target material needs to be further sorted later.

[0090] S703: If the current quantities corresponding to all target materials match the corresponding material quantities, the material sorting operation corresponding to the target product is completed.

[0091] Specifically, if the current quantities corresponding to all target materials match the corresponding material quantities, it means that there are no target materials that need to be further sorted, and it can be determined that the material sorting operation corresponding to the target product has been completed. For example, the sorting controller 10 can receive sorting order information corresponding to K target products, and the sorting order information corresponding to each target product includes N target materials, and the number of materials of each target material is M. If the current quantities corresponding to the N target materials reach the corresponding material quantities, it is determined that the material sorting operation corresponding to one target product is completed.

[0092] It can be understood that when the sorting order information acquired by the sorting controller 10 is specifically the sorting order information corresponding to at least one target product, that is, K≥1; then after step S703, the product sorting processing method based on three-dimensional vision also includes: updating the product quantity corresponding to the target product, and judging whether the product quantity reaches the order quantity (that is, K) corresponding to the target product in the sorting order information; if the product quantity reaches the order quantity, it is determined that the material sorting operation corresponding to the sorting order information is completed; if the product quantity does not reach the order quantity, it is necessary to repeat steps S701-S703 until the product quantity reaches the order quantity, so as to realize the material sorting operation according to the flexible sorting order information.

[0093] S704: If the current quantity corresponding to the target material does not match the corresponding material quantity, and there is no identification and positioning information corresponding to the unsorted target material, repeatedly control the three-dimensional vision mechanism to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool, and obtain the identification and positioning information corresponding to the target material identified based on the three-dimensional image of the material.

[0094] Specifically, if the current quantity corresponding to the target material does not match the material quantity, it means that there are target materials that need to be further sorted. At this time, it is necessary to determine whether there is any identification and positioning information corresponding to the unsorted target material in the identification and positioning information corresponding to the target material identified by the three-dimensional image of the material collected by the three-dimensional vision mechanism 30; if there is no identification and positioning information corresponding to the unsorted target material, it is necessary to repeatedly control the three-dimensional vision mechanism to collect the material three-dimensional image corresponding to the material placed on the sorting pool, and obtain the identification and positioning information corresponding to the target material identified based on the material three-dimensional image, that is, repeat step S602 and subsequent steps to re-collect the material three-dimensional image and identify the corresponding identification and positioning information, and re-sort the material based on the re-acquired identification and positioning information. It can be understood that by controlling the three-dimensional vision mechanism 30 to re-collect the material three-dimensional image and identify the corresponding identification and positioning information, the material three-dimensional image corresponding to the material on the sorting pool 21 can be collected in layers, and the materials can be prevented from being blocked or stacked on each other to affect the smooth progress of material sorting.

[0095] As an optimization scheme of step S704, step S704 specifically includes: if the current quantity corresponding to the target material does not match the corresponding material quantity, and there is no identification and positioning information corresponding to the unsorted target material, control the sorting motion unit to flip the material placed on the sorting pool 21, and repeatedly control the three-dimensional vision mechanism 30 to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool 21, and obtain the identification and positioning information corresponding to the target material identified based on the three-dimensional image of the material. It can be understood that before the three-dimensional vision mechanism 30 needs to be controlled to re-collect the three-dimensional image of the material and identify the corresponding identification and positioning information, the sorting motion unit of the sorting pool mechanism 20 is controlled to move to flip the material placed on the sorting pool 21, improve the material flipping efficiency, and reduce the occurrence of materials blocking or stacking each other, which helps to improve the material sorting efficiency.

[0096] As an example, if the current quantity corresponding to the target material does not match the corresponding material quantity, and there is no identification and positioning information corresponding to the unsorted target material, the sorting controller 10 determines whether the flipping condition is met according to the identification and positioning information corresponding to the currently collected three-dimensional image of the material. If the flipping condition is met, the sorting motion unit is controlled to flip the material placed on the sorting pool 21, and the three-dimensional visual mechanism 30 is repeatedly controlled to collect the material three-dimensional image corresponding to the material placed on the sorting pool 21, and the identification and positioning information corresponding to the target material identified based on the material three-dimensional image is obtained. Among them, the flipping condition is a condition pre-set for controlling the sorting pool mechanism to flip the material. The flipping condition can be that the unsorted quantity corresponding to the unsorted target material in the currently collected three-dimensional image of the material is less than the preset quantity threshold, or the category of the unsorted target material in the currently collected three-dimensional image of the material does not match the category of the target material in the sorting order information, or other pre-set conditions. It can be understood that when the pre-set flipping condition is met, the sorting motion unit is automatically controlled to flip the material placed on the sorting pool 21, so as to improve the material flipping efficiency and reduce the occurrence of materials blocking or stacking each other, which is helpful to improve the material sorting efficiency.

[0097] S704: If the current quantity corresponding to the target material does not match the corresponding material quantity, and there is identification and positioning information corresponding to the unsorted target material, repeatedly control the sorting robot to pick up the target material from the sorting pool to the corresponding receiving port according to the identification and positioning information corresponding to the target material.

[0098] Specifically, if there is a situation where the current quantity corresponding to the target material does not match the material quantity, it means that there is a target material that needs to be further sorted. At this time, it is necessary to determine whether there is any identification and positioning information corresponding to the unsorted target material in the identification and positioning information corresponding to the target material identified by the three-dimensional image of the material collected by the three-dimensional vision mechanism 30; if there is identification and positioning information of the unsorted target material, the identification and positioning information corresponding to the target material can be repeatedly executed to control the sorting robot to pick up the target material from the sorting pool to the corresponding material receiving port, that is, repeat step S701 and subsequent steps until all target materials corresponding to the target product for assembling identified by the three-dimensional image of the same material are sorted to the corresponding material receiving port 42, so as to reduce the workload of the three-dimensional vision mechanism 30 for image acquisition and identification and positioning processing, which helps to improve the sorting processing efficiency.

[0099] In the product sorting and processing method based on three-dimensional vision provided in this embodiment, based on the identification and positioning information with relatively accurate identification and positioning results, the sorting robot 41 is controlled to pick up the target material from the sorting pool 21 to the corresponding material receiving port 42, which helps to ensure the smooth progress of the sorting work. The current quantity corresponding to each target material is updated in real time. If the current quantity corresponding to all target materials matches the corresponding material quantity, the sorting operation of all target materials corresponding to the order sorting information is completed. If the current quantity corresponding to the target material does not match the corresponding material quantity and there is no identification and positioning information corresponding to the unsorted target material, the three-dimensional vision mechanism 30 is controlled to re-collect the three-dimensional image of the material and identify the corresponding identification and positioning information. The material is sorted again based on the re-acquired identification and positioning information, which can realize the layered collection of the three-dimensional image of the material corresponding to the material on the sorting pool 21, and can avoid the mutual occlusion or stacking between materials to affect the smooth progress of material sorting. If the current quantity corresponding to the target material does not match the corresponding material quantity and there is identification and positioning information corresponding to the unsorted target material, the sorting robot 41 is repeatedly controlled to sort the materials to reduce the workload of the three-dimensional vision mechanism 30 for image acquisition and identification and positioning processing, which helps to improve the sorting processing efficiency.

[0100] In one embodiment, if Figure 8 As shown, the identification and positioning information includes position information and posture information. Accordingly, step S701, that is, according to the identification and positioning information corresponding to the target material, controlling the sorting robot 41 to pick up the target material from the sorting pool 21 to the corresponding receiving port 42, specifically includes the following steps:

[0101] S801: Determine the target robot according to the position information and the sorting area corresponding to each sorting robot.

[0102] Specifically, when there are at least two sorting robots 41, the sorting area of ​​each sorting robot 41 in the pre-built three-dimensional space coordinate system shared by the three-dimensional vision mechanism 30 and the sorting execution mechanism 40 can be pre-configured. After obtaining the position information (x, y, z) and posture information (α, θ, φ) corresponding to each target material, the sorting controller 10 matches the sorting area corresponding to each sorting robot 41 based on the position information (x, y, z) to determine the sorting robot 41 that can pick up the target material as the target robot, so as to perform orderly sorting control based on the sorting areas corresponding to at least two sorting robots 41. The sorting robot 41 can pick up the target material identified in its sorting area to the corresponding receiving port 42 according to the target material and material quantity in the sorting order information to complete the sorting operation of the target material.

[0103] S802: According to the posture information, the picking angle of the target robot is adjusted to pick up the target material from the sorting pool to the corresponding receiving port.

[0104] Specifically, when determining the target robot corresponding to each target material, the sorting controller 10 can adjust the picking angle of the target robot according to the posture information (α, θ, φ) corresponding to the target material, so as to accurately pick up the target material from the sorting pool 21 to the corresponding material receiving port 42, thereby ensuring the smooth progress of the material sorting process, achieving the effect of flexible sorting of target materials with uncertain postures, and improving the sorting efficiency of the target materials.

[0105] In the product sorting and processing method based on three-dimensional vision provided in the present embodiment, the target robot corresponding to the target material is first determined based on the position information in the identification and positioning information and the sorting area corresponding to each sorting robot 41, so as to ensure the sorting efficiency of the target robot in sorting the target material; and then, based on the posture information in the identification and positioning information, the picking angle of the target robot for picking up the target material is adjusted to achieve the effect of flexible sorting of target materials with uncertain postures, thereby improving the sorting efficiency of the target materials.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A product sorting and processing system based on three-dimensional vision, It is characterized in that include: Sorting controller; The sorting pool mechanism comprises a sorting pool for placing materials; the sorting pool mechanism also comprises a sorting motion unit connected to the sorting pool and the sorting controller, and is used to flip the materials placed on the sorting pool under the control of the sorting controller; the sorting pool comprises a sorting column and at least one sorting ring arranged on the periphery of the sorting column; the sorting ring is a circular cylinder, and the height of the inner ring of the circular cylinder is less than the height of the outer ring; the sorting motion unit is connected to the sorting column, and is used to control the sorting column to reciprocate along the axial direction of the sorting column; the sorting motion unit is connected to the sorting ring, and is used to control the sorting ring to reciprocate along the axial direction of the sorting ring; A three-dimensional vision mechanism is arranged above the sorting pool and connected to the sorting controller, and is used to collect a three-dimensional image of a material corresponding to the material placed on the sorting pool under the control of the sorting controller, identify identification and positioning information corresponding to the target material based on the three-dimensional image of the material, and send the identification and positioning information to the sorting controller; A sorting execution mechanism is arranged at the periphery of the sorting pool and connected to the sorting controller, and is used for sorting materials according to the identification and positioning information under the control of the sorting controller.

2. The product sorting and processing system based on three-dimensional vision as claimed in claim 1, It is characterized in that The three-dimensional vision mechanism includes a three-dimensional camera and a vision processing unit. The three-dimensional camera is arranged directly above the sorting pool and is used to collect a three-dimensional image of the material corresponding to the material placed on the sorting pool; the vision processing unit is connected to the three-dimensional camera and is used to identify the identification and positioning information corresponding to the target material based on the three-dimensional image of the material.

3. The product sorting and processing system based on three-dimensional vision as claimed in claim 2, It is characterized in that The three-dimensional vision mechanism also includes a vision light source, which is arranged directly above the sorting pool and directly below the three-dimensional camera.

4. The product sorting and processing system based on three-dimensional vision as claimed in claim 1, It is characterized in that The sorting execution mechanism includes at least one material receiving port, and the sorting controller is connected to at least one sorting robot arranged outside the sorting pool, sends material sorting instructions to the sorting robot according to the identification and positioning information, and receives the target material through the material receiving port.

5. A product sorting and processing method based on three-dimensional vision, comprising the following steps performed by a sorting controller: Acquire sorting order information, wherein the sorting order information includes target materials; The three-dimensional vision mechanism is controlled to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool, and obtain the identification and positioning information corresponding to the target material identified based on the three-dimensional image of the material; the sorting pool mechanism includes a sorting motion unit connected to the sorting pool and the sorting controller, which is used to flip the material placed on the sorting pool under the control of the sorting controller; the sorting pool includes a sorting column and at least one sorting ring arranged on the periphery of the sorting column; the sorting motion unit is connected to the sorting column, and is used to control the sorting column to reciprocate along the axial direction of the sorting column; the sorting motion unit is connected to the sorting ring, and is used to control the sorting ring to reciprocate along the axial direction of the sorting ring; The sorting execution mechanism is controlled to sort materials according to the identification and positioning information.

6. The product sorting method based on three-dimensional vision as claimed in claim 5, It is characterized in that The sorting order information includes the quantity of materials corresponding to the target materials; The controlling sorting execution mechanism performs material sorting according to the identification and positioning information, including: According to the identification and positioning information corresponding to the target material, the sorting robot is controlled to pick up the target material from the sorting pool to the corresponding material receiving port; Updating the current quantity corresponding to each of the target materials, and determining whether the current quantities corresponding to all of the target materials match the corresponding material quantities; If the current quantities corresponding to all the target materials match the corresponding material quantities, the material sorting operation corresponding to the target product is completed.

7. The product sorting method based on three-dimensional vision as claimed in claim 6, It is characterized in that After updating the current quantity corresponding to each of the target materials and determining whether the current quantities corresponding to all the target materials match the corresponding material quantities, the product sorting processing method further includes: If the current quantity corresponding to the target material does not match the corresponding material quantity, and there is no identification and positioning information corresponding to the unsorted target material, repeatedly controlling the three-dimensional vision mechanism to collect the three-dimensional image of the material corresponding to the material placed on the sorting pool, and obtaining the identification and positioning information corresponding to the target material identified based on the three-dimensional image of the material; or If the current quantity corresponding to the target material does not match the corresponding material quantity, and there is identification and positioning information corresponding to the unsorted target material, repeatedly execute the identification and positioning information corresponding to the target material to control the sorting robot to pick up the target material from the sorting pool to the corresponding material receiving port.

8. The product sorting method based on three-dimensional vision as claimed in claim 6, It is characterized in that The identification and positioning information includes position information and posture information; According to the identification and positioning information corresponding to the target material, controlling the sorting robot to pick up the target material from the sorting pool to the corresponding material receiving port includes: Determine a target robot according to the position information and a sorting area corresponding to each of the sorting robots; According to the posture information, the picking angle of the target robot is adjusted to pick up the target material from the sorting pool to the corresponding material receiving port.

Images