A method and system for generating sorting tasks based on image recognition

Abstract

The present invention is applicable to the field of intelligent sorting technology, and in particular, relates to a method and system for generating sorting tasks based on image recognition, the method comprising: constructing an intelligent sorting system, constructing a sorting system model based on the intelligent sorting system; running the intelligent sorting system, acquiring cargo monitoring images, performing shape analysis on the cargo monitoring images, and generating cargo entity models; identifying cargo label information, determining a sorting path, simulating a conveying path according to the sorting path, and determining a sorting time; reading the initial state of a sorting robot arm, generating a robot arm sorting task, and sending the robot arm sorting task to a corresponding sorting robot arm. The present invention independently controls each sorting robot arm, sends control instructions to each sorting robot arm, and can realize the operation of multiple sorting robot arms through a data processing center, thereby improving sorting efficiency, and can realize the sorting of cargo while using high-cost intelligent robot arms, thereby reducing operating costs.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent sorting, and in particular relates to a method and system for generating sorting tasks based on image recognition. Background Art

[0002] Intelligent sorting is an efficient logistics solution that combines automated equipment with information technology. It uses advanced identification technologies (such as barcode scanning, RFID radio frequency identification, etc.), automatic control systems, and mechanical devices to achieve rapid and accurate classification and transportation of items. This system can significantly improve the work efficiency of warehouses or distribution centers, reduce manual errors, and flexibly adjust sorting strategies according to order requirements. It is suitable for e-commerce, manufacturing, postal services and other industries. With the development of artificial intelligence and Internet of Things technologies, the intelligence level and application scope of intelligent sorting systems are constantly expanding.

[0003] In the existing warehousing and logistics transportation process, the sorting of some goods mainly relies on manual work. The manual sorting method is inefficient and prone to missed sorting problems. Summary of the invention

[0004] The purpose of the present invention is to provide a method for generating sorting tasks based on image recognition, aiming to solve the problem of low sorting efficiency and easy omission of sorting when manually sorting boxes.

[0005] The present invention is implemented as follows: a method for generating a sorting task based on image recognition, the method comprising:

[0006] Constructing an intelligent sorting system, and constructing a sorting system model based on the intelligent sorting system, wherein the intelligent sorting system includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices, and at least one group of sorting mechanical arms, and the data processing center is connected with the image acquisition device and the sorting mechanical arm;

[0007] Run the intelligent sorting system, obtain cargo monitoring images, perform shape analysis on the cargo monitoring images, and generate cargo entity models in the sorting system model;

[0008] Based on the cargo monitoring image, the cargo label information is identified, the sorting path is determined, the conveying path is simulated according to the sorting path, and the sorting time to reach the corresponding sorting robot arm is determined;

[0009] Read the initial state of the sorting robot arm, generate the robot arm sorting task according to the posture information of the cargo entity model, and send the robot arm sorting task to the corresponding sorting robot arm.

[0010] Preferably, the steps of operating the intelligent sorting system, acquiring cargo monitoring images, performing shape analysis on the cargo monitoring images, and generating cargo entity models in the sorting system model specifically include:

[0011] The intelligent sorting system is operated to continuously detect images through two sets of image acquisition devices, and temporarily store all images recorded from the time when the image changes to the time when the change ends;

[0012] Performing image preprocessing on all images to obtain preprocessed images, retaining images that meet preset screening conditions in the preprocessed images, and obtaining cargo monitoring images;

[0013] The edge size of the goods is identified based on the goods monitoring image, and a physical model of the goods is generated in the sorting system model according to the edge size.

[0014] Preferably, the steps of identifying cargo label information based on cargo monitoring images, determining a sorting path, simulating a conveying path according to the sorting path, and determining a sorting time for reaching a corresponding sorting robot arm specifically include:

[0015] Extract the barcode area in the cargo monitoring image, perform image recognition on the barcode area, and obtain cargo label information;

[0016] Identify the type of goods according to the goods label information, perform pre-sorting according to the type of goods, determine the direction of the goods, and obtain the sorting path;

[0017] Calculate the sorting time when the current goods arrive at the corresponding sorting robot arm according to the sorting path.

[0018] Preferably, the steps of reading the initial state of the sorting robot arm, generating a robot arm sorting task according to the posture information of the cargo entity model, and sending the robot arm sorting task to the corresponding sorting robot arm specifically include:

[0019] Read the initial state of the sorting robot arm, determine the robot arm's sorting preparatory posture according to the posture information of the cargo entity model, and generate preparatory action control instructions;

[0020] Determine the sorting state of the sorting robot arm, determine the movement path of the gripper of the sorting robot arm based on the sorting state of the sorting robot arm, and determine the gripping action control instruction according to the movement path of the gripper;

[0021] A robot arm sorting task is generated based on the preparatory action control instructions and the gripping action control instructions, and the sorting robot arm grips and sorts the goods at a specified time according to the robot arm sorting task.

[0022] Preferably, two groups of image acquisition devices acquire two groups of cargo monitoring images, and the image acquisition viewing angles of the two image acquisition devices are respectively downward shooting and shooting along the horizontal plane, and the preset screening condition is: in the cargo monitoring image obtained by overhead shooting, there is only one surface of the box body corresponding to the cargo, and at this time, the image acquisition direction of the image acquisition device used for overhead shooting is directly above the cargo.

[0023] Another object of the present invention is to provide a system for generating sorting tasks based on image recognition, the system comprising:

[0024] A model building module, used for building an intelligent sorting system, and building a sorting system model based on the intelligent sorting system, wherein the intelligent sorting system includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices, and at least one group of sorting mechanical arms, and the data processing center is connected with the image acquisition device and the sorting mechanical arms;

[0025] A cargo model generation module is used to run the intelligent sorting system, obtain cargo monitoring images, perform shape analysis on the cargo monitoring images, and generate a cargo entity model in the sorting system model;

[0026] The conveying simulation module is used to identify the cargo label information based on the cargo monitoring image, determine the sorting path, simulate the conveying path according to the sorting path, and determine the sorting time to reach the corresponding sorting robot arm;

[0027] The sorting task generation module is used to read the initial state of the sorting robot arm, generate the robot arm sorting task according to the posture information of the cargo entity model, and send the robot arm sorting task to the corresponding sorting robot arm.

[0028] Preferably, the cargo model generation module includes:

[0029] The cargo monitoring unit is used to operate the intelligent sorting system. It continuously detects images through two sets of image acquisition devices and temporarily stores all images recorded from the time when the image changes to the time when the image changes;

[0030] An image screening unit is used to perform image preprocessing on all images to obtain preprocessed images, retain images that meet preset screening conditions in the preprocessed images, and obtain cargo monitoring images;

[0031] The model building unit is used to identify the edge size of the goods based on the goods monitoring image, and generate a physical model of the goods in the sorting system model according to the edge size.

[0032] Preferably, the transport simulation module comprises:

[0033] A cargo information extraction unit is used to extract the barcode area in the cargo monitoring image, perform image recognition on the barcode area, and obtain cargo label information;

[0034] A sorting path generating unit is used to identify the type of goods according to the goods label information, perform pre-sorting according to the type of the goods, determine the direction of the goods, and obtain a sorting path;

[0035] The sorting time calculation unit is used to calculate the sorting time for the current goods to arrive at the corresponding sorting robot arm according to the sorting path.

[0036] Preferably, the sorting task generating module includes:

[0037] A preparatory instruction generating unit is used to read the initial state of the sorting robot arm, determine the preparatory posture of the robot arm for sorting according to the posture information of the cargo entity model, and generate a preparatory action control instruction;

[0038] A gripping instruction generating unit, used to determine the sorting state of the sorting robot arm, determine the gripper movement path of the sorting robot arm based on the sorting state of the sorting robot arm, and determine the gripping action control instruction according to the gripper movement path;

[0039] The task generation unit is used to generate a robot arm sorting task based on the preparatory action control instruction and the gripping action control instruction. The sorting robot arm grips and sorts the goods at a specified time according to the robot arm sorting task.

[0040] Preferably, two groups of image acquisition devices acquire two groups of cargo monitoring images, and the image acquisition viewing angles of the two image acquisition devices are respectively downward shooting and shooting along the horizontal plane, and the preset screening condition is: in the cargo monitoring image obtained by overhead shooting, there is only one surface of the box body corresponding to the cargo, and at this time, the image acquisition direction of the image acquisition device used for overhead shooting is directly above the cargo.

[0041] The present invention provides a method for generating sorting tasks based on image recognition, which can identify continuous goods through an image acquisition device set up at a single location, and then independently control each sorting robot arm, send control instructions to each sorting robot arm, and realize the operation of multiple sorting robot arms through a data processing center, thereby improving the sorting efficiency, and can realize the sorting of goods while using high-cost intelligent robot arms, thereby reducing the operating cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a method for generating sorting tasks based on image recognition provided by an embodiment of the present invention;

[0043] Figure 2 A flowchart of the steps of operating an intelligent sorting system, acquiring a cargo monitoring image, performing shape analysis on the cargo monitoring image, and generating a cargo entity model in a sorting system model provided by an embodiment of the present invention;

[0044] Figure 3A flowchart of the steps of identifying cargo label information based on cargo monitoring images, determining a sorting path, simulating a conveying path according to the sorting path, and determining a sorting time to reach a corresponding sorting robot arm provided by an embodiment of the present invention;

[0045] Figure 4 A flowchart of the steps of reading the initial state of a sorting robot arm, generating a robot arm sorting task according to the posture information of a cargo entity model, and sending the robot arm sorting task to a corresponding sorting robot arm provided in an embodiment of the present invention;

[0046] Figure 5 An architecture diagram of a system for generating sorting tasks based on image recognition provided by an embodiment of the present invention;

[0047] Figure 6 An architecture diagram of a cargo model generation module provided by an embodiment of the present invention;

[0048] Figure 7 An architecture diagram of a transport simulation module provided by an embodiment of the present invention;

[0049] Figure 8 An architecture diagram of a sorting task generation module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] like Figure 1 FIG. 1 is a flowchart of a method for generating a sorting task based on image recognition provided by an embodiment of the present invention, the method comprising:

[0052] S100, constructing an intelligent sorting system, and constructing a sorting system model based on the intelligent sorting system, wherein the intelligent sorting system includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices and at least one group of sorting robotic arms, and the data processing center is connected to the image acquisition device and the sorting robotic arms.

[0053] In this step, an intelligent sorting system is constructed. The intelligent sorting system includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices and at least one group of sorting mechanical arms. The conveyor belt is used to transport goods. There are multiple sorting mechanical arms, all of which are arranged on both sides of the conveyor belt. The sorting mechanical arms adopt four-axis mechanical arms or five-axis mechanical arms. The sorting mechanical arms are used to move according to the sorting tasks issued by the data processing center. There are two groups of image acquisition devices, one of which is located directly above the conveyor belt and is used to take a bird's-eye view of the goods to obtain a bird's-eye view image. At the same time, the other group of image acquisition devices is arranged on the side of the conveyor belt, so as to capture images of the goods from the side, thereby obtaining images from two perspectives for determining the shape and size of the goods. The data processing center is used to receive the images captured by the image acquisition device, identify the size of the goods by performing image processing, and construct a three-dimensional model according to the installation position of each part in the intelligent sorting system. The position and state of the conveyor belt, the image acquisition device and the sorting mechanical arm are determined in the three-dimensional model to obtain a sorting system model. The sorting system model is updated in real time according to the state of the sorting mechanical arm and the state of the goods.

[0054] S200, running the intelligent sorting system, acquiring cargo monitoring images, performing shape analysis on the cargo monitoring images, and generating cargo entity models in the sorting system model.

[0055] In this step, the intelligent sorting system is run, and the goods are placed on the conveyor belt. The conveyor belt drives the goods through the position where the image acquisition device is located. The image acquisition device monitors the real-time picture. When the picture does not change, no image acquisition is performed, and monitoring is continued. When the picture changes, it indicates that there is a goods passing by. At this time, image acquisition is performed. Each cargo will correspond to multiple images. One image is selected from the images taken by each image acquisition device as the cargo monitoring image. Therefore, the cargo monitoring image includes at least two images, one is a top-down perspective, and the other is a side-view perspective. Based on the two sets of cargo monitoring images, the outer dimensions of the cargo are determined according to the number of pixels occupied by the cargo in the picture, and then a cargo entity model is generated in the sorting system model according to the outer dimensions.

[0056] S300, identifying cargo label information based on cargo monitoring images, determining a sorting path, simulating a conveying path according to the sorting path, and determining a sorting time to reach a corresponding sorting robot arm.

[0057] In this step, the cargo label information is identified based on the cargo monitoring image. A cargo label is affixed to the cargo. The cargo label may be a QR code or a bar code for identification. By scanning the cargo label, the identity of the current cargo and the subsequent transportation direction can be quickly identified, thereby determining the sorting path and transporting it to different sorting robotic arms for sorting. The status of the conveyor belt, that is, the transport speed of the conveyor belt, is obtained in real time. According to the current position of the cargo and the transport speed of the conveyor belt, the time for the cargo to arrive at each sorting robotic arm can be calculated, that is, the sorting time is obtained.

[0058] S400, reading the initial state of the sorting robot arm, generating a robot arm sorting task according to the posture information of the cargo entity model, and sending the robot arm sorting task to the corresponding sorting robot arm.

[0059] In this step, the initial state of the sorting robot arm is read. Due to the different sizes and placement directions of the goods, the sorting robot arm needs to be adjusted to different states to adapt to the current goods. The sorting robot arm includes a multi-axis host and a clamp. The multi-axis host can be four-axis or five-axis. The clamp can use two parallel clamps. The two sets of parallel clamps move relative to each other to achieve clamping of the goods. The clamping state of the sorting robot arm when clamping can be determined according to the placement posture of the goods. The robot arm sorting task is generated according to the initial state and clamping state of the sorting robot arm. The robot arm sorting task records the rotation angle of each axis of the sorting robot arm during operation and the placement posture of the clamp, which is sent to the sorting robot arm. The sorting robot arm runs according to the robot arm sorting task to achieve clamping of the goods. During the whole process, when identifying the goods, a robot arm sorting task formulated for the corresponding sorting robot arm is generated, so that the sorting robot arm can clamp the goods without the assistance of visual recognition. That is, the present invention can be used for commonly used low-cost programmable robot arms, which can realize the operation process of intelligent robot arms, greatly reducing the production cost.

[0060] like Figure 2 As shown, as a preferred embodiment of the present invention, the steps of operating the intelligent sorting system, acquiring cargo monitoring images, performing shape analysis on the cargo monitoring images, and generating cargo entity models in the sorting system model specifically include:

[0061] S201, running the intelligent sorting system, continuously performing image detection through two groups of image acquisition devices, and temporarily storing all images recorded from the time when the image changes to the time when the change ends.

[0062] In this step, the intelligent sorting system is operated, the conveyor belt is started, and the goods are placed on the conveyor belt one by one, and the goods are transported by the conveyor belt. Two groups of image acquisition devices are fixedly installed on a rack set on the conveyor belt. The image acquired by the image acquisition device for overhead shooting is the first image, and the image acquired by the image acquisition device for side shooting is the second image. When the two groups of image acquisition devices are working, image acquisition is performed at a preset time interval, such as image acquisition is performed every 50ms, and two adjacent groups of images are compared to compare pixel information. When the gray value difference of the pixel is greater than the preset value, it is determined that the pixel has changed. When the number of pixel changes in the picture exceeds the ratio, it is determined that the picture has changed. When the picture changes, image recording begins until the picture no longer changes. All images during this period are saved, that is, each image acquisition device will record a series of images in one detection cycle (from one picture change to no longer changing the picture).

[0063] S202, performing image preprocessing on all images to obtain preprocessed images, retaining images that meet preset screening conditions in the preprocessed images, and obtaining cargo monitoring images.

[0064] In this step, all images are preprocessed. Specifically, the images are gray-scaled and converted into gray-scale images. The gray-scale images are further processed. All straight lines in the image are extracted by performing Hough linear transformation. Since the packages of the goods are all rectangular structures, the position of the goods can be determined according to the lines in the image. The preset screening conditions are: in the cargo monitoring image obtained by overhead shooting, there is only one surface of the box corresponding to the goods. At this time, the image acquisition direction of the image acquisition device for overhead shooting is located directly above the goods. Then, if there is only one rectangle in the image after Hough linear transformation, the current image is retained and used as the cargo monitoring image corresponding to the image acquisition device for overhead shooting (overhead monitoring image), the time when the image acquisition device acquires the image is extracted, and the image taken at the corresponding time is extracted from the image acquired by the image acquisition device for side shooting, and it is used as the cargo monitoring image acquired by the image acquisition device for side shooting (side shooting monitoring image).

[0065] S203, identifying the edge size of the goods based on the goods monitoring image, and generating a goods entity model in the sorting system model according to the edge size.

[0066] In this step, the edge size of the cargo is identified based on the cargo monitoring image. The two sets of image acquisition devices have the same configuration, that is, they use the same industrial camera and the same shooting parameters. The actual length corresponding to each pixel in the picture is determined according to the preset scale. In the overhead monitoring image, the four edge lines on the top of the cargo are determined. The above edge lines record the length and width of the cargo. The number of pixels contained in the cargo in the length and width directions is determined. The length and width of the cargo are determined according to the scale. The projection length of the rectangle where the cargo in the overhead monitoring image is located in the plane where the side monitoring image is located is determined, and then the image of the cargo in the side monitoring image is extracted. At this time, the edge of the cargo in the side monitoring image is a rectangle, and the lengths of the upper and lower sides of the rectangle are equal to the projection length. The height of the cargo is determined according to the aspect ratio of the rectangle corresponding to the cargo in the side monitoring image. At this time, the length, width and height of the cargo are all confirmed. Since the installation positions of the two sets of image acquisition devices are known, the position of the cargo in the sorting system model is also known, so as to generate a cargo entity model, and the cargo entity model is consistent with the actual posture state of the cargo on the conveyor belt.

[0067] like Figure 3 As shown, as a preferred embodiment of the present invention, the steps of identifying cargo label information based on cargo monitoring images, determining a sorting path, simulating a conveying path according to the sorting path, and determining the sorting time to reach a corresponding sorting robot arm specifically include:

[0068] S301, extracting the barcode area in the cargo monitoring image, performing image recognition on the barcode area, and obtaining cargo label information.

[0069] In this step, the barcode area in the cargo monitoring image is extracted. During the cargo transportation process, it is necessary to mark the cargo information by pasting identification marks such as QR codes. By performing image acquisition, the label information can be read to determine the cargo label information.

[0070] S302, identifying the type of goods according to the goods label information, performing pre-sorting according to the type of the goods, determining the direction of the goods, and obtaining a sorting path.

[0071] In this step, the type of goods is identified based on the cargo label information. The cargo label information records the type and specific information of the goods. Based on the above information, the sorting path of the goods can be queried, that is, which conveyor belts the goods pass through and which sorting robot arm they are sent to for sorting.

[0072] S303, calculating the sorting time for the current goods to arrive at the corresponding sorting robot arm according to the sorting path.

[0073] In this step, the sorting time for the current goods to arrive at the corresponding sorting robot arm is calculated according to the sorting path. During the entire operation of the conveyor belt, the data processing center can obtain the transportation speed of the conveyor belt in real time. Then, according to the time when the goods pass through the image acquisition device, the time when the goods arrive at the corresponding sorting robot arm can be calculated according to the positional relationship between the conveyor belt and the sorting robot arm. Before this time, the sorting robot arm needs to adjust the state of the gripper in advance to prepare to clamp the goods.

[0074] like Figure 4 As shown, as a preferred embodiment of the present invention, the steps of reading the initial state of the sorting robot arm, generating the robot arm sorting task according to the posture information of the cargo entity model, and sending the robot arm sorting task to the corresponding sorting robot arm specifically include:

[0075] S401, reading the initial state of the sorting robot arm, determining the robot arm sorting preparation posture according to the posture information of the cargo entity model, and generating a preparation action control instruction.

[0076] In this step, the initial state of the sorting robot arm is read. The initial state is the final state of the sorting robot arm after the sorting robot arm completes the last cargo sorting. Of course, the final states of all sorting robot arms can be set to be consistent, that is, after the sorting robot arm completes the sorting, it returns to a fixed position and state. The robot arm's sorting preparatory posture is determined according to the posture information of the cargo entity model. The sorting preparatory posture is set according to the state of the cargo. The clamping jaws in the sorting preparatory posture are parallel to a pair of planes of the cargo, so that when clamping, the clamping jaws move downward and move in the direction of the cargo movement to complete the clamping of the cargo, and generate preparatory action control instructions. The preparatory action control instructions are used to control the clamping jaws to run to a specified height and opening width.

[0077] S402, determining the sorting state of the sorting robot arm, determining the movement path of the gripper of the sorting robot arm based on the sorting state of the sorting robot arm, and determining the gripping action control instruction according to the movement path of the gripper.

[0078] S403, generating a robot arm sorting task based on the preparatory action control instruction and the gripping action control instruction, and the sorting robot arm grips and sorts the goods at a specified time according to the robot arm sorting task.

[0079] In this step, the sorting state of the sorting robot arm is determined. The sorting state is the position of the gripper and the sorting robot arm when the gripper successfully grips the goods. Then, the control instruction for the gripper, that is, the gripping action control instruction, is constructed according to the action stroke in this process. Similarly, according to the initial state and the sorting state of the sorting robot arm, the running path of the sorting robot arm in this process is determined to generate the corresponding control instruction. All the obtained control instructions are sent to the sorting robot arm as a robot arm sorting task. The sorting robot arm executes according to the time and action parameters specified in the robot arm sorting task, so as to realize the gripping and sorting of the goods. In the whole process of the present invention, only one image acquisition is performed to automatically generate the image for controlling the ordinary multi-axis robot arm to work. A group of image acquisition devices can be used to realize independent control of multiple sorting robot arms, which greatly reduces the investment cost while ensuring the sorting efficiency.

[0080] like Figure 5 As shown, a system for generating sorting tasks based on image recognition is provided in an embodiment of the present invention, and the system includes:

[0081] The model building module 100 is used to build an intelligent sorting system, and to build a sorting system model based on the intelligent sorting system. The intelligent sorting system includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices and at least one group of sorting robotic arms. The data processing center is connected to the image acquisition device and the sorting robotic arms.

[0082] In this system, the model building module 100 builds an intelligent sorting system, which includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices and at least one group of sorting mechanical arms. The conveyor belt is used to transport goods. There are multiple sorting mechanical arms, all of which are arranged on both sides of the conveyor belt. The sorting mechanical arms adopt four-axis mechanical arms or five-axis mechanical arms. The sorting mechanical arms are used to move according to the sorting tasks issued by the data processing center. There are two groups of image acquisition devices, one of which is located directly above the conveyor belt and is used to take a bird's-eye view of the goods to obtain a bird's-eye view image. At the same time, the other group of image acquisition devices is arranged on the side of the conveyor belt, so as to capture images of the goods from the side, thereby obtaining images from two perspectives for determining the shape and size of the goods. The data processing center is used to receive the images captured by the image acquisition device, identify the size of the goods by performing image processing, and construct a three-dimensional model according to the installation positions of various parts in the intelligent sorting system. The positions and states of the conveyor belt, the image acquisition device and the sorting mechanical arms are determined in the three-dimensional model to obtain a sorting system model. The sorting system model is updated in real time according to the state of the sorting mechanical arms and the state of the goods.

[0083] The cargo model generation module 200 is used to run the intelligent sorting system, obtain cargo monitoring images, perform shape analysis on the cargo monitoring images, and generate a cargo entity model in the sorting system model.

[0084] In this system, the cargo model generation module 200 runs the intelligent sorting system, places the cargo on the conveyor belt, and drives the cargo through the position of the image acquisition device through the conveyor belt. The image acquisition device monitors the real-time picture. When the picture does not change, no image acquisition is performed, and monitoring is continued. When the picture changes, it indicates that there is cargo passing by. At this time, image acquisition is performed. Each cargo will correspond to multiple images. One image is selected from the images taken by each image acquisition device as the cargo monitoring image. Therefore, the cargo monitoring image includes at least two images, one is a top-down perspective, and the other is a side-view perspective. Based on the two sets of cargo monitoring images, the external dimensions of the cargo are determined according to the number of pixels occupied by the cargo in the picture, and then a cargo entity model is generated in the sorting system model according to the external dimensions.

[0085] The transport simulation module 300 is used to identify cargo label information based on cargo monitoring images, determine a sorting path, simulate a transport path according to the sorting path, and determine a sorting time to reach a corresponding sorting robot arm.

[0086] In this system, the transport simulation module 300 identifies the cargo label information based on the cargo monitoring image. A cargo label is affixed to the cargo. The cargo label may be a QR code or a bar code for identification. By scanning the cargo label, the identity of the current cargo and the subsequent transportation direction can be quickly identified, thereby determining the sorting path and transporting it to different sorting robotic arms for sorting. The status of the conveyor belt, that is, the transport speed of the conveyor belt, is obtained in real time. According to the current position of the cargo and the transport speed of the conveyor belt, the time for the cargo to arrive at each sorting robotic arm can be calculated, that is, the sorting time is obtained.

[0087] The sorting task generation module 400 is used to read the initial state of the sorting robot arm, generate the robot arm sorting task according to the posture information of the cargo entity model, and send the robot arm sorting task to the corresponding sorting robot arm.

[0088] In this system, the sorting task generation module 400 reads the initial state of the sorting robot arm. Due to the different sizes and placement directions of the goods, the sorting robot arm needs to be adjusted to different states to adapt to the current goods. The sorting robot arm includes a multi-axis host and a clamp. The multi-axis host can be a four-axis or a five-axis. The clamp can use two parallel clamps. The two sets of parallel clamps move relative to each other to achieve the clamping of the goods. The clamping state of the sorting robot arm when clamping can be determined according to the placement posture of the goods. The robot arm sorting task is generated according to the initial state and clamping state of the sorting robot arm. Picking task, the robot arm sorting task records the rotation angle of each axis of the sorting robot arm during operation and the placement posture of the gripper, and sends it to the sorting robot arm. The sorting robot arm operates according to the robot arm sorting task to achieve the gripping of the goods. During the whole process, when the goods are identified, a robot arm sorting task formulated for the corresponding sorting robot arm is generated, so that the sorting robot arm can grip the goods without the assistance of visual recognition. That is, the present invention can be used for commonly used low-cost programmable robot arms, which can realize the operation process of intelligent robot arms, greatly reducing production costs.

[0089] like Figure 6 As shown, as a preferred embodiment of the present invention, the cargo model generation module 200 includes:

[0090] The cargo monitoring unit 201 is used to operate the intelligent sorting system, continuously detect images through two sets of image acquisition devices, and temporarily store all images recorded from the time when the image changes to the time when the image changes.

[0091] In this module, the cargo monitoring unit 201 runs the intelligent sorting system, starts the conveyor belt, places the cargo one by one on the conveyor belt, and transports the cargo through the conveyor belt. Two groups of image acquisition devices are fixedly installed on a rack set on the conveyor belt. The image acquired by the image acquisition device for overhead shooting is the first image, and the image acquired by the image acquisition device for side shooting is the second image. When the two groups of image acquisition devices are working, image acquisition is performed at a preset time interval, such as performing image acquisition once every 50ms, and comparing the two adjacent groups of images to compare pixel information. When the gray value difference of the pixel is greater than the preset value, it is determined that the pixel has changed. When the number of pixel changes in the picture exceeds the ratio, it is determined that the picture has changed. When the picture changes, image recording begins until the picture no longer changes. All images during this period are saved, that is, each image acquisition device will record a series of images in one detection cycle (from one picture change to no longer changing the picture).

[0092] The image screening unit 202 is used to perform image preprocessing on all images to obtain preprocessed images, retain images that meet preset screening conditions in the preprocessed images, and obtain cargo monitoring images.

[0093] In this module, the image screening unit 202 performs image preprocessing on all images. Specifically, the image is gray-processed to convert it into a gray-scale image, and the gray-scale image is further processed. By performing Hough linear transformation, all straight lines in the image are extracted. Since the packages of the goods are all rectangular structures, the position of the goods can be determined according to the lines in the image. The preset screening conditions are: in the cargo monitoring image obtained by overhead shooting, there is only one surface of the box corresponding to the goods. At this time, the image acquisition direction of the image acquisition device used for overhead shooting is located directly above the goods. Then, if there is only one rectangle in the image after Hough linear transformation, the current image is retained and used as the cargo monitoring image corresponding to the image acquisition device used for overhead shooting (overhead monitoring image), the time when the image acquisition device acquires the image is extracted, and the image taken at the corresponding time is extracted from the image acquired by the image acquisition device used for side shooting, and it is used as the cargo monitoring image acquired by the image acquisition device used for side shooting (side shooting monitoring image).

[0094] The model building unit 203 is used to identify the edge size of the goods based on the goods monitoring image, and generate a goods entity model in the sorting system model according to the edge size.

[0095] In this module, the model building unit 203 identifies the edge size of the cargo based on the cargo monitoring image. The two groups of image acquisition devices have the same configuration, that is, they use the same industrial camera and the same shooting parameters. The actual length corresponding to each pixel in the picture is determined according to the preset scale. In the overhead monitoring image, the four edge lines on the top of the cargo are determined. The above edge lines record the length and width of the cargo. The number of pixels contained in the cargo in the length and width directions is determined. The length and width of the cargo are determined according to the scale. The projection length of the rectangle where the cargo in the overhead monitoring image is located in the plane where the side monitoring image is located is determined, and then the image of the cargo in the side monitoring image is extracted. At this time, the edge of the cargo in the side monitoring image is a rectangle, and the lengths of the upper and lower sides of the rectangle are equal to the projection length. Then, according to the aspect ratio of the rectangle corresponding to the cargo in the side monitoring image, the height of the cargo is determined. At this time, the length, width and height of the cargo are all confirmed. Since the installation positions of the two groups of image acquisition devices are known, the position of the cargo in the sorting system model is also known, so as to generate a cargo entity model, and the cargo entity model is consistent with the actual posture state of the cargo on the conveyor belt.

[0096] like Figure 7 As shown, as a preferred embodiment of the present invention, the transport simulation module 300 includes:

[0097] The cargo information extraction unit 301 is used to extract the barcode area in the cargo monitoring image, perform image recognition on the barcode area, and obtain cargo label information.

[0098] In this module, the cargo information extraction unit 301 extracts the barcode area in the cargo monitoring image. During the cargo transportation process, it is necessary to mark the cargo information by pasting identification marks such as QR codes. By performing image acquisition, the label information can be read to determine the cargo label information.

[0099] The sorting path generating unit 302 is used to identify the type of goods according to the goods label information, perform pre-sorting according to the type of the goods, determine the direction of the goods, and obtain the sorting path.

[0100] In this module, the sorting path generation unit 302 identifies the type of goods based on the goods label information. The goods label information records the type and specific information of the goods. Based on the above information, the sorting path of the goods can be queried, that is, which conveyor belts the goods pass through and which sorting robot arm they are sent to for sorting.

[0101] The sorting time calculation unit 303 is used to calculate the sorting time for the current goods to arrive at the corresponding sorting robot arm according to the sorting path.

[0102] In this module, the sorting time calculation unit 303 calculates the sorting time for the current goods to arrive at the corresponding sorting robot arm according to the sorting path. During the entire operation of the conveyor belt, the data processing center can obtain the transportation speed of the conveyor belt in real time. Then, according to the time when the goods pass through the image acquisition device, the time when the goods arrive at the corresponding sorting robot arm can be calculated according to the positional relationship between the conveyor belt and the sorting robot arm. Before this time, the sorting robot arm needs to adjust the state of the gripper in advance to prepare to grip the goods.

[0103] like Figure 8 As shown, as a preferred embodiment of the present invention, the sorting task generating module 400 includes:

[0104] The preparatory instruction generating unit 401 is used to read the initial state of the sorting robot arm, determine the preparatory posture of the robot arm for sorting according to the posture information of the cargo entity model, and generate a preparatory action control instruction.

[0105] In this module, the preparatory instruction generating unit 401 reads the initial state of the sorting robot arm, which is the final state of the sorting robot arm after the sorting robot arm completes the last cargo sorting. Of course, the final states of all sorting robot arms can be set to be consistent, that is, after the sorting robot arm completes the sorting, it returns to a fixed position and state, and the robot arm sorting preparatory posture is determined according to the posture information of the cargo entity model. The sorting preparatory posture is set according to the state of the cargo. The clamping jaws in the sorting preparatory posture are parallel to a pair of planes of the cargo, so that when clamping, the clamping jaws move downward and move in the direction of the cargo movement to complete the clamping of the cargo, and generate preparatory action control instructions. The preparatory action control instructions are used to control the clamping jaws to run to a specified height and opening width.

[0106] The gripping instruction generating unit 402 is used to determine the sorting state of the sorting robot arm, determine the gripper movement path of the sorting robot arm based on the sorting state of the sorting robot arm, and determine the gripping action control instruction according to the gripper movement path.

[0107] The task generation unit 403 is used to generate a robot arm sorting task based on the preparatory action control instruction and the gripping action control instruction. The sorting robot arm grips and sorts the goods at a specified time according to the robot arm sorting task.

[0108] In this module, the sorting state of the sorting robot arm is determined. The sorting state is the position of the gripper and the sorting robot arm when the gripper successfully grips the goods. Then, the control instruction for the gripper is constructed according to the action stroke in this process, that is, the gripping action control instruction. Similarly, according to the initial state and the sorting state of the sorting robot arm, the running path of the sorting robot arm in this process is determined to generate the corresponding control instruction. All the obtained control instructions are sent to the sorting robot arm as a robot arm sorting task. The sorting robot arm executes according to the time and action parameters specified in the robot arm sorting task, so as to realize the gripping and sorting of the goods. In the whole process of the present invention, only one image acquisition is performed to automatically generate the image for controlling the ordinary multi-axis robot arm to work. A group of image acquisition devices can be used to realize independent control of multiple sorting robot arms, which greatly reduces the investment cost while ensuring the sorting efficiency.

[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating sorting tasks based on image recognition, characterized in that: The method comprises: Constructing an intelligent sorting system, and constructing a sorting system model based on the intelligent sorting system, wherein the intelligent sorting system includes a conveyor belt, a group of data processing centers, two groups of image acquisition devices, and at least one group of sorting mechanical arms, and the data processing center is connected with the image acquisition device and the sorting mechanical arm; Run the intelligent sorting system, obtain cargo monitoring images, perform shape analysis on the cargo monitoring images, and generate cargo entity models in the sorting system model; Based on the cargo monitoring image, the cargo label information is identified, the sorting path is determined, the conveying path is simulated according to the sorting path, and the sorting time to reach the corresponding sorting robot arm is determined; Read the initial state of the sorting robot arm, generate the robot arm sorting task according to the posture information of the cargo entity model, and send the robot arm sorting task to the corresponding sorting robot arm; The steps of operating the intelligent sorting system, acquiring cargo monitoring images, performing shape analysis on the cargo monitoring images, and generating cargo entity models in the sorting system model specifically include: The intelligent sorting system is operated to continuously detect images through two sets of image acquisition devices, and temporarily store all images recorded from the time when the image changes to the time when the change ends; Performing image preprocessing on all images to obtain preprocessed images, retaining images that meet preset screening conditions in the preprocessed images, and obtaining cargo monitoring images; Identify the edge size of the goods based on the goods monitoring image, and generate a physical model of the goods in the sorting system model according to the edge size; The intelligent sorting system is operated, the conveyor belt is started, and the goods are placed on the conveyor belt one by one, and the goods are transported by the conveyor belt. Two sets of image acquisition devices are fixedly installed on the rack set on the conveyor belt. The image acquired by the image acquisition device for overhead shooting is the first image, and the image acquired by the image acquisition device for side shooting is the second image. When the two sets of image acquisition devices are working, image acquisition is performed at a preset time interval, and two adjacent sets of images are compared to compare pixel information. When the gray value difference of the pixel is greater than the preset value, it is determined that the pixel has changed. When the number of pixel changes in the picture exceeds the ratio, it is determined that the picture has changed. When the picture changes, image recording begins until the picture no longer changes. All images during this period are saved, that is, each image acquisition device will record a series of images in one detection cycle. The picture is gray-processed to convert it into a gray-scale image, and the gray-scale image is further processed. By performing Hough linear transformation, all straight lines in the picture are extracted. Since the packages of the goods are all rectangular structures, the position of the goods is determined according to the lines in the picture. The preset screening conditions are: in the cargo monitoring image obtained by overhead shooting, there is only one face of the box corresponding to the goods. At this time, the image acquisition direction of the image acquisition device used for overhead shooting is located directly above the goods. When there is only one rectangle in the image after Hough linear transformation, the current image is retained and used as the cargo monitoring image corresponding to the image acquisition device used for overhead shooting, that is, the overhead monitoring image. The time when the image acquisition device acquired the image is extracted, and the image taken at the corresponding time is extracted from the image acquired by the image acquisition device used for side shooting, and it is used as the cargo monitoring image acquired by the image acquisition device used for side shooting, that is, the side shooting monitoring image; The two sets of image acquisition devices have the same configuration, use the same industrial camera and the same shooting parameters, and determine the actual length corresponding to each pixel in the picture according to the preset scale. In the overhead monitoring image, determine the four edge lines on the top of the goods. The edge lines record the length and width of the goods. Determine the number of pixels contained in the length and width directions of the goods, determine the length and width of the goods according to the scale, and determine the projection length of the rectangle where the goods in the overhead monitoring image are located in the plane where the side monitoring image is located, and then extract the image of the goods in the side monitoring image. At this time, the edge of the picture of the goods in the side monitoring image is a rectangle, and the length of the upper and lower sides of the rectangle is equal to the projection length. Then, according to the aspect ratio of the rectangle corresponding to the goods in the side monitoring image, determine the height of the goods. At this time, the length, width and height of the goods are confirmed. Since the installation positions of the two sets of image acquisition devices are known, the position of the goods in the sorting system model is obtained, and a physical model of the goods is generated.

2. The method for generating sorting tasks based on image recognition according to claim 1, characterized in that: The steps of identifying cargo label information based on cargo monitoring images, determining a sorting path, simulating a conveying path according to the sorting path, and determining a sorting time for reaching a corresponding sorting robot arm specifically include: Extract the barcode area in the cargo monitoring image, perform image recognition on the barcode area, and obtain cargo label information; Identify the type of goods according to the goods label information, perform pre-sorting according to the type of goods, determine the direction of the goods, and obtain the sorting path; Calculate the sorting time when the current goods arrive at the corresponding sorting robot arm according to the sorting path.

3. The method for generating sorting tasks based on image recognition according to claim 1, characterized in that: The steps of reading the initial state of the sorting robot arm, generating a robot arm sorting task according to the position information of the cargo entity model, and sending the robot arm sorting task to the corresponding sorting robot arm specifically include: Read the initial state of the sorting robot arm, determine the robot arm's sorting preparatory posture according to the posture information of the cargo entity model, and generate preparatory action control instructions; Determine the sorting state of the sorting robot arm, determine the movement path of the gripper of the sorting robot arm based on the sorting state of the sorting robot arm, and determine the gripping action control instruction according to the movement path of the gripper; A robot arm sorting task is generated based on the preparatory action control instructions and the gripping action control instructions, and the sorting robot arm grips and sorts the goods at a specified time according to the robot arm sorting task.

4. The method for generating sorting tasks based on image recognition according to claim 1, characterized in that: Two groups of image acquisition devices acquire two groups of cargo monitoring images. The image acquisition viewing angles of the two image acquisition devices are respectively downward shooting and shooting along the horizontal plane. The preset screening condition is: in the cargo monitoring image obtained by overhead shooting, there is only one surface of the box corresponding to the cargo. At this time, the image acquisition direction of the image acquisition device used for overhead shooting is directly above the cargo.

Images