Robot cargo unloading method and system based on target identification

Abstract

The invention discloses a robot cargo unloading method and system based on target recognition. The method comprises the following steps: acquiring an image of a to-be-unloaded cargo by using an image acquisition module; carrying out segmentation identification on the goods based on a preset shape; the goods are unloaded through the grabbing module; if image recognition fails, a radio frequency emission module arranged on the grabbing module is started, and the position of the cargo is determined according to a passive light-emitting RFID tag arranged on the cargo; and cargo information is obtained according to the passive light-emitting RFID tag, and the grabbing module strategy is adjusted to unload the cargo. The automatic identification of the cargos can be effectively realized, and meanwhile, the reliability of cargo unloading is guaranteed by various identification strategies.

Description

Technical field:

[0001] The present invention relates to robot control, and more particularly to a robot cargo unloading method and system based on target recognition. Background technology:

[0002] At present, with the steady and rapid development of the domestic commodity market, the demand for logistics is increasing. The efficiency of the backward method of using manpower to carry, load and unload goods to stack goods is far from meeting the explosive growth demand of the current logistics industry. At the same time, labor costs are rising rapidly. Carrying and loading and unloading goods are labor-intensive activities, especially in some harsh environments and when the goods are toxic and corrosive, the damage to human health will be very large.

[0003] The intelligence level of my country's unloading robots is not high. Most of them are pre-programmed and then complete repetitive single operations at designated locations. They are unable to self-adjust in unknown dynamic industrial environments to adapt to changes in the environment and operating objects. How to use robots to autonomously identify and load and unload goods has become an urgent problem to be solved. Summary of the invention:

[0004] In view of the defects of insufficient intelligence and single function of the existing unloading robots, this application associates the robot's image acquisition module with the grabbing module; uses the image acquisition module to obtain the image of the goods to be unloaded, and pre-processes the image of the goods to be unloaded; segment and identify the goods based on the preset shape; use the grabbing module to unload the goods; if the image recognition fails, turn on the radio frequency transmission module set in the grabbing module, and determine the position of the goods according to the passive luminous RFID tag set on the goods, wherein the luminous RFID tag is set at the center of the top surface or side of the goods; obtain the goods information according to the passive luminous RFID tag, and adjust the grabbing module strategy to unload the goods. It can effectively solve the problem of autonomous identification of goods, and at the same time, multiple identification strategies ensure the reliability of goods unloading.

[0006] The technical solution adopted by the present invention to solve the above technical problems is:

[0007] The robot cargo unloading method based on target recognition comprises the following steps:

[0008] S1, associating the robot's image acquisition module with the grasping module;

[0009] S11, the image acquisition module includes an RGB-D camera, the camera internal parameters are calibrated through a machine pinhole model, the relative position relationship between the color camera and the depth camera is calibrated, and a calibration system is established;

[0010] S12, constructing a grabbing module coordinate system, determining a coordinate transformation matrix, and associating the calibration system with the grabbing module coordinate system;

[0011] S2. Using an image acquisition module to acquire an image of the cargo to be unloaded, and preprocessing the image of the cargo to be unloaded;

[0012] S3, segmenting and identifying the goods based on the preset shape;

[0013] S4, unloading the goods using the grabbing module;

[0014] S5. If the image recognition fails, the radio frequency transmission module set in the grabbing module is turned on to determine the position of the goods according to the passive luminous RFID tag set on the goods, wherein the luminous RFID tag is set at the center of the top surface or side surface of the goods;

[0015] S6. Obtain cargo information according to the passive luminous RFID tag and adjust the grab module strategy to unload the cargo.

[0016] Furthermore, the pretreatment process includes

[0017] S21, graying the image of the cargo to be unloaded;

[0018] S22, filtering the grayscale image;

[0019] S23, superimposing and fusing two adjacent frames of images and then outputting them;

[0020] The grayscale processing is calculated as follows:

[0021] gray(i,j)=0.3R(i,j)+0.58G(i,j)+0.12B(i,j),

[0022] Wherein gray() represents the gray value, R() represents the red channel value, G() represents the green channel value, B() represents the blue channel value, i and j are positive integers representing the pixel position; the filtering includes: first Gaussian filtering Where m represents the number of pixels in the neighborhood and then the median filtering operation is performed.

[0023] Furthermore, the segmentation and identification includes:

[0024] S31, extracting a set of straight line elements in the image of the cargo to be unloaded using a straight line detection and segmentation algorithm,

[0025] S32, after filtering the straight line element set by using the first length threshold, merging the adjacent line segments by using the merging model, and extracting the merged straight line contour;

[0026] S33, performing cargo segmentation according to the similarity and area ratio between the preset shape and the straight line contour.

[0027] Furthermore, the merging model is:

[0028] d min <D,d max >length li ,d max >length lj

[0029] d min =min(d1,d2,d3,d4), d max =max(d1, d2, d3, d4), d1, d2, d3, d4 are the distances between the coordinates of the endpoints of adjacent line segments, D represents the second threshold, and length represents the length of the line segment;

[0030] If the contour line is an unclosed figure, a circumscribed closed figure is set according to a preset figure.

[0031] Furthermore, in the segmentation and identification of goods based on preset shapes, the preset shapes include squares, triangles, and rectangles.

[0032] Furthermore, the segmentation and identification of goods based on a preset shape also includes: segmenting the image based on the shape of the image color connected area and texture features.

[0033] Furthermore, in the process of acquiring cargo information according to the passive luminous RFID tag and adjusting the grabbing module strategy to unload the cargo, the cargo information includes the mass and size of the cargo.

[0034] A robot cargo unloading system based on target recognition, comprising:

[0035] An initialization module, used to associate the robot's image acquisition module with the grasping module;

[0036] An image acquisition module, used to obtain an image of the cargo to be unloaded by using the image acquisition module, and pre-process the image of the cargo to be unloaded;

[0037] An image segmentation module is used to segment and identify goods based on preset shapes;

[0038] An operating module, used for unloading the goods using the grabbing module;

[0039] A backup identification module is used to activate the radio frequency transmission module provided in the grabbing module if the image recognition fails, and determine the position of the goods according to the passive luminous RFID tag provided on the goods, wherein the luminous RFID tag is provided at the center of the top surface or side surface of the goods;

[0040] The secondary operation module is used to obtain cargo information according to the passive luminous RFID tag and adjust the grabbing module strategy to unload the cargo.

[0041] The system also includes a preprocessing module, which is used to:

[0042] S91, graying the image of the cargo to be unloaded;

[0043] S92, filtering the grayscale image;

[0044] S93, superimposing and fusing two adjacent frames of images and then outputting them.

[0045] In the image segmentation module:

[0046] S101, extracting a set of straight line elements in the image of the cargo to be unloaded using a straight line detection and segmentation algorithm,

[0047] S102, after filtering the straight line element set by using the first length threshold, merging the adjacent line segments by using a merging model to extract the merged straight line contour;

[0048] S103 , performing cargo segmentation according to the similarity and area ratio between the preset shape and the straight line contour.

[0049] The beneficial effects of the present invention are as follows:

[0050] The robot recognition method in this application can enable the robot to accurately identify and locate the goods in a complex industrial carriage environment to achieve accurate grasping, high efficiency, automation, streamline logistics lines, reduce costs, and improve production efficiency. It can effectively solve the problem of autonomous identification of goods, and at the same time, multiple identification strategies ensure the reliability of cargo unloading.

[0051] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically cited and described in detail as follows. Description of the drawings:

[0052] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0053] Figure 1 Flowchart of the robotic cargo unloading method based on object recognition. Specific implementation method:

[0054] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0055] In the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be connected, detachably connected, or integrated; it can be mechanically connected or electrically connected; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] Example 1

[0057] The robot cargo unloading method based on target recognition comprises the following steps:

[0058] S1, associating the robot's image acquisition module with the grasping module;

[0059] S11, the image acquisition module includes an RGB-D camera, the camera internal parameters are calibrated through a machine pinhole model, the relative position relationship between the color camera and the depth camera is calibrated, and a calibration system is established;

[0060] S12, constructing a grabbing module coordinate system, determining a coordinate transformation matrix, and associating the calibration system with the grabbing module coordinate system;

[0061] S2. Using an image acquisition module to acquire an image of the cargo to be unloaded, and preprocessing the image of the cargo to be unloaded;

[0062] S3, segmenting and identifying the goods based on the preset shape;

[0063] S4, unloading the goods using the grabbing module;

[0064] S5. If the image recognition fails, the radio frequency transmission module set in the grabbing module is turned on to determine the position of the goods according to the passive luminous RFID tag set on the goods, wherein the luminous RFID tag is set at the center of the top surface or side surface of the goods;

[0065] S6. Obtain cargo information according to the passive luminous RFID tag and adjust the grab module strategy to unload the cargo.

[0066] Furthermore, the pretreatment process includes

[0067] S21, graying the image of the cargo to be unloaded;

[0068] S22, filtering the grayscale image;

[0069] S23, superimposing and fusing two adjacent frames of images and then outputting them;

[0070] The grayscale processing is calculated as follows:

[0071] gray(i,j)=0.3R(i,j)+0.58G(i,j)+0.12B(i,j),

[0072] Wherein gray() represents the gray value, R() represents the red channel value, G() represents the green channel value, B() represents the blue channel value, i and j are positive integers representing the pixel position; the filtering includes: first Gaussian filtering Where m represents the number of pixels in the neighborhood and then the median filtering operation is performed.

[0073] Furthermore, the segmentation and identification includes:

[0074] S31, extracting a set of straight line elements in the image of the cargo to be unloaded using a straight line detection and segmentation algorithm,

[0075] S32, after filtering the straight line element set by using the first length threshold, merging the adjacent line segments by using the merging model, and extracting the merged straight line contour;

[0076] S33, performing cargo segmentation according to the similarity and area ratio between the preset shape and the straight line contour.

[0077] Furthermore, the merging model is:

[0078] d min <D,d max >length li ,d max >length lj

[0079] d min =min(d1,d2,d3,d4), d max =max(d1, d2, d3, d4), d1, d2, d3, d4 are the distances between the coordinates of the endpoints of adjacent line segments, D represents the second threshold, and length represents the length of the line segment;

[0080] If the contour line is an unclosed figure, a circumscribed closed figure is set according to a preset figure.

[0081] Furthermore, in the segmentation and identification of goods based on preset shapes, the preset shapes include squares, triangles, and rectangles.

[0082] Furthermore, the segmentation and identification of goods based on a preset shape also includes: segmenting the image based on the shape of the image color connected area and texture features.

[0083] Furthermore, in the process of acquiring cargo information according to the passive luminous RFID tag and adjusting the grabbing module strategy to unload the cargo, the cargo information includes the mass and size of the cargo.

[0084] Example 2

[0085] A robot cargo unloading system based on target recognition, comprising:

[0086] An initialization module, used to associate the robot's image acquisition module with the grasping module;

[0087] An image acquisition module, used to obtain an image of the cargo to be unloaded by using the image acquisition module, and pre-process the image of the cargo to be unloaded;

[0088] An image segmentation module is used to segment and identify goods based on preset shapes;

[0089] An operating module, used for unloading the goods using the grabbing module;

[0090] A backup identification module is used to activate the radio frequency transmission module provided in the grabbing module if the image recognition fails, and determine the position of the goods according to the passive luminous RFID tag provided on the goods, wherein the luminous RFID tag is provided at the center of the top surface or side surface of the goods;

[0091] The secondary operation module is used to obtain cargo information according to the passive luminous RFID tag and adjust the grabbing module strategy to unload the cargo.

[0092] The system also includes a preprocessing module, which is used to:

[0093] S91, graying the image of the cargo to be unloaded;

[0094] S92, filtering the grayscale image;

[0095] S93, superimposing and fusing two adjacent frames of images and then outputting them.

[0096] In the image segmentation module:

[0097] S101, extracting a set of straight line elements in the image of the cargo to be unloaded using a straight line detection and segmentation algorithm,

[0098] S102, after filtering the straight line element set by using the first length threshold, merging the adjacent line segments by using a merging model to extract the merged straight line contour;

[0099] S103 , performing cargo segmentation according to the similarity and area ratio between the preset shape and the straight line contour.

[0100] The advantages of the present invention are:

[0101] The robot recognition method in this application can enable the robot to accurately identify and locate the goods in a complex industrial carriage environment to achieve accurate grasping, high efficiency, automation, streamline logistics lines, reduce costs, and improve production efficiency. It can effectively solve the problem of autonomous identification of goods, and multiple identification strategies ensure the reliability of cargo unloading.

[0102] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A robot cargo unloading method based on target recognition, characterized in that: The following steps are involved: S1, associating the robot's image acquisition module with the grasping module; S11, the image acquisition module includes an RGB-D camera, the camera internal parameters are calibrated through a machine pinhole model, the relative position relationship between the color camera and the depth camera is calibrated, and a calibration system is established; S12, constructing a grabbing module coordinate system, determining a coordinate transformation matrix, and associating the calibration system with the grabbing module coordinate system; S2. Using an image acquisition module to acquire an image of the cargo to be unloaded, and preprocessing the image of the cargo to be unloaded; S3, segmenting and identifying the goods based on the preset shape; S4, unloading the goods using the grabbing module; S5. If the image recognition fails, the radio frequency transmission module set in the grabbing module is turned on to determine the position of the goods according to the passive luminous RFID tag set on the goods, wherein the luminous RFID tag is set at the center of the top surface or side surface of the goods; S6. Obtain cargo information according to the passive luminous RFID tag and adjust the grab module strategy to unload the cargo.

2. The robot cargo unloading method based on target recognition according to claim 1 is characterized in that: The pretreatment process includes S21, graying the image of the cargo to be unloaded; S22, filtering the grayscale image; S23, superimposing and fusing two adjacent frames of images and then outputting them; The grayscale processing is calculated as follows: gray(i,j)=0.3R(i,j)+0.58G(i,j)+0.12B(i,j), Wherein gray() represents the gray value, R() represents the red channel value, G() represents the green channel value, B() represents the blue channel value, i and j are positive integers representing the pixel position; the filtering includes: first Gaussian filtering Where m represents the number of pixels in the neighborhood and then the median filtering operation is performed.

3. The robot cargo unloading method based on target recognition according to claim 1 is characterized in that: The segmentation identification includes: S31, extracting a set of straight line elements in the image of the cargo to be unloaded using a straight line detection and segmentation algorithm, S32, after filtering the straight line element set by using the first length threshold, merging the adjacent line segments by using the merging model, and extracting the merged straight line contour; S33, performing cargo segmentation according to the similarity and area ratio between the preset shape and the straight line contour.

4. The robot cargo unloading method based on target recognition according to claim 3 is characterized in that: The merge model is: d min <D,d max >length li ,d max >length lj d min =min(d1,d2,d3,d4), d max =max(d1, d2, d3, d4), d1, d2, d3, d4 are the distances between the coordinates of the endpoints of adjacent line segments, D represents the second threshold, and length represents the length of the line segment; If the contour line is an unclosed figure, a circumscribed closed figure is set according to a preset figure.

5. The robot cargo unloading method based on target recognition according to claim 1 is characterized in that: In the segmentation and identification of goods based on preset shapes, the preset shapes include squares, triangles, and rectangles.

6. The robot cargo unloading method based on target recognition according to claim 1 is characterized in that: The segmentation and identification of goods based on a preset shape also includes: segmenting the image based on the shape of the image color connected area and texture features.

7. The robot cargo unloading method based on target recognition according to claim 1 is characterized in that: In the process of acquiring cargo information according to the passive luminous RFID tag and adjusting the grabbing module strategy to unload the cargo, the cargo information includes the mass and size of the cargo.

8. A robot cargo unloading system based on target recognition, characterized in that: An initialization module, used to associate the robot's image acquisition module with the grasping module; An image acquisition module, used to obtain an image of the cargo to be unloaded by using the image acquisition module, and pre-process the image of the cargo to be unloaded; An image segmentation module is used to segment and identify goods based on preset shapes; An operating module, used for unloading the goods using the grabbing module; A backup identification module is used to activate the radio frequency transmission module provided in the grabbing module if the image recognition fails, and determine the position of the goods according to the passive luminous RFID tag provided on the goods, wherein the luminous RFID tag is provided at the center of the top surface or side surface of the goods; The secondary operation module is used to obtain cargo information according to the passive luminous RFID tag and adjust the grabbing module strategy to unload the cargo.

9. The robot cargo unloading system based on target recognition according to claim 8, characterized in that: The system also includes a preprocessing module, which is used to: S91, graying the image of the cargo to be unloaded; S92, filtering the grayscale image; S93, superimposing and fusing two adjacent frames of images and then outputting them.

10. The robot cargo unloading system based on target recognition according to claim 8, characterized in that: In the image segmentation module: S101, extracting a set of straight line elements in the image of the cargo to be unloaded using a straight line detection and segmentation algorithm, S102, after filtering the straight line element set by using the first length threshold, merging the adjacent line segments by using a merging model to extract the merged straight line contour; S103 , performing cargo segmentation according to the similarity and area ratio between the preset shape and the straight line contour.

Images