Robot control method, device and equipment

Abstract

The present disclosure provides a robot control method, device, and apparatus. The method comprises: controlling the robot to move to a second position corresponding to a first position where a box to be picked up is located, and then controlling the robot to obtain at least one image of the shelf at the first position; controlling the robot's pick-up and placement device to move based on the image information, so as to detect whether there is an obstacle in the pick-up and placement device's telescopic direction using a ranging sensor on the pick-up and placement device; and controlling the robot's pick-up and placement device to pick the box to be picked up if no obstacle exists. This method can determine in real time whether there is an obstacle in the robot's picking direction and control the robot to pick up the box based on the determination result, thereby improving the robot's safety and the success rate of picking up the box.

Description

[0001] This application is a divisional application with application number 202210296519.3, application date March 24, 2022, and invention name “Robot control method, device and equipment”. Technical Field

[0002] The present disclosure relates to the field of intelligent warehousing technology, and in particular to a robot control method, device, and equipment. Background Art

[0003] With the development of technology, robots have gradually replaced human labor and are being used in warehousing and logistics systems. In warehousing and logistics systems, robots are no longer just responsible for handling tasks, but also for tasks such as picking up and releasing goods.

[0004] In the prior art, the dispatching system controls the robot to go to the corresponding location and pick up the goods based on the stored location information of the material box or the location information of the storage location.

[0005] Since the scheduling system cannot obtain the loading details on the shelves in real time, it cannot determine in real time whether there are obstacles in the robot's operating area on the shelves, which may cause interference when the robot picks and places goods, resulting in the robot being unable to complete the task smoothly and even threatening the robot's safety. Summary of the Invention

[0006] The present disclosure provides a robot control method, device and equipment, which can determine in real time whether there are obstacles in the robot's picking direction, and control the robot to pick up goods based on the judgment result, thereby improving the robot's safety and the success rate of picking up goods.

[0007] In a first aspect, the present disclosure provides a robot control method, which is applied to a control device, and the method includes: controlling the robot to move to a second position corresponding to a first position where a box to be picked up is located, and then controlling the robot to obtain image information of at least one shelf where the first position is located; based on the image information, controlling the robot's picking and placing device to move, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, controlling the robot's picking and placing device to perform a picking operation on the box to be picked up.

[0008] Optionally, the robot's picking and placing device is controlled to move according to the image information, including: obtaining the length information of the cargo box along the extension and retraction direction of the picking and placing device; obtaining the field of view angle of the ranging sensor; and controlling the robot's picking and placing device to move according to the image information, length information and field of view angle.

[0009] Optionally, the robot's picking and placing device is controlled to move according to the image information, length information and field of view angle, including: determining a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf according to the image information, the first surface being the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; controlling the robot's picking and placing device to move according to the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and length information.

[0010] Optionally, controlling the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and the length information includes: obtaining the position information of the ranging sensor; determining the height difference between the ranging sensor and the shelf based on the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; controlling the robot's picking and placing device to move based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection point of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, the second surface being the surface of the cargo box opposite to the robot's ranging sensor and away from the ranging sensor.

[0011] Optionally, based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, the robot's picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, including: determining the target height difference based on the first distance, the field of view angle and the length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; controlling the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0012] Optionally, it also includes: after controlling the robot to move to a third position corresponding to the target storage location, controlling the robot to obtain image information of at least one shelf where the target storage location is located; based on the image information, determining whether there is an obstacle on the target storage location, and if there is no obstacle, controlling the robot's picking and placing device to perform a placing operation on the target storage location.

[0013] Optionally, the image information is a two-dimensional image, and the robot is controlled to obtain image information of at least one shelf where the target storage location is located, including: after controlling the robot's picking and placing device to be raised to a preset height, the robot is controlled to obtain image information of at least one shelf where the target storage location is located; accordingly, based on the image information, determining whether there is an obstacle at the target storage location, including: determining whether there is an obstacle at the target storage location based on whether the image information includes identification information of the cargo box.

[0014] Optionally, the image information is a two-dimensional image, and determining whether there is an obstacle at the target storage location based on the image information includes: inputting the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0015] Optionally, the image information is a three-dimensional image, and determining whether there is an obstacle at the target storage location based on the image information includes: parsing the image information to obtain a distribution of noise points in the image information; and determining whether there is an obstacle at the target storage location based on the noise point distribution.

[0016] Optionally, controlling the robot to obtain at least one image information of the shelf at the first position includes: controlling the robot's picking and placing device to rise and fall to the fourth position based on the acquired height information of the cargo box, and then controlling the robot to obtain at least one image information of the shelf at the first position.

[0017] In a second aspect, the present disclosure also provides a robot control method, which is applied to the robot, the robot including a picking and placing device, an image acquisition unit and a ranging sensor, the ranging sensor being arranged on the picking and placing device, the method including: responding to a first control instruction sent by a control device, moving to a second position corresponding to a first position where a box to be picked up is located, and acquiring image information of at least one shelf at the first position through the image acquisition unit; sending the image information to the control device; responding to a second control instruction sent by the control device, controlling the picking and placing device to move so that the control device detects whether there is an obstacle in the extension and retraction direction of the picking and placing device through the ranging sensor on the picking and placing device; responding to a third control instruction sent by the control device, controlling the picking and placing device to perform a picking operation on the box to be picked up, the third control instruction being sent by the control device when it detects that there is no obstacle in the extension and retraction direction of the picking and placing device.

[0018] Optionally, it also includes: receiving a fourth control instruction sent by the control device, the fourth control instruction is generated and sent by the control device based on the first distance, the height difference between the ranging sensor and the shelf, the field of view of the ranging sensor and the length information of the cargo box along the telescopic direction of the picking and placing device, the first distance is the distance between the ranging sensor and the plane where the first surface of the cargo box is located; according to the fourth control instruction, the picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection, the target intersection is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0019] Optionally, in response to a fifth control instruction sent by the control device, after the cargo picking and placing device is controlled to rise and fall to the fourth position, at least one image information of the shelf at the first position is obtained through the image acquisition unit. The fifth control instruction is generated and sent by the control device based on the obtained height information of the cargo box.

[0020] In a third aspect, the present disclosure also provides a robot control method, which is applied to a control device. The method includes: controlling the robot to move to a third position corresponding to a target storage location, and then controlling the robot to obtain image information of at least one shelf where the target storage location is located; determining whether there is an obstacle on the target storage location based on the image information; if there is no obstacle, controlling the robot's picking and placing device to perform a placing operation on the target storage location.

[0021] Optionally, the image information is a two-dimensional image, and the robot is controlled to obtain image information of at least one shelf where the target storage location is located, including: after controlling the robot's picking and placing device to be raised to a preset height, the robot is controlled to obtain image information of at least one shelf where the target storage location is located; accordingly, based on the image information, determining whether there is an obstacle at the target storage location, including: determining whether there is an obstacle at the target storage location based on whether the image information includes identification information of the cargo box.

[0022] Optionally, the image information is a two-dimensional image, and determining whether there is an obstacle at the target storage location based on the image information includes: inputting the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0023] Optionally, the image information is a three-dimensional image, and determining whether there is an obstacle at the target storage location based on the image information includes: parsing the image information to obtain a distribution of noise points in the image information; and determining whether there is an obstacle at the target storage location based on the noise point distribution.

[0024] Optionally, it also includes: after controlling the robot to move to a second position corresponding to the first position where the box to be picked up is located, controlling the robot to obtain image information of at least one shelf where the first position is located; according to the image information, controlling the robot's picking and placing device to move, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, controlling the robot's picking and placing device to perform a picking operation on the box to be picked up.

[0025] Optionally, the robot's picking and placing device is controlled to move according to the image information, including: obtaining the length information of the cargo box along the extension and retraction direction of the picking and placing device; obtaining the field of view angle of the ranging sensor; and controlling the robot's picking and placing device to move according to the image information, length information and field of view angle.

[0026] Optionally, the robot's picking and placing device is controlled to move according to the image information, length information and field of view angle, including: determining a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf according to the image information, the first surface being the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; controlling the robot's picking and placing device to move according to the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and length information.

[0027] Optionally, controlling the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and the length information includes: obtaining the position information of the ranging sensor; determining the height difference between the ranging sensor and the shelf based on the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; controlling the robot's picking and placing device to move based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection point of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, the second surface being the surface of the cargo box opposite to the robot's ranging sensor and away from the ranging sensor.

[0028] Optionally, based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, the robot's picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, including: determining the target height difference based on the first distance, the field of view angle and the length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; controlling the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0029] Optionally, controlling the robot to obtain at least one image information of the shelf at the first position includes: controlling the robot's picking and placing device to rise and fall to the fourth position based on the acquired height information of the cargo box, and then controlling the robot to obtain at least one image information of the shelf at the first position.

[0030] In a fourth aspect, the present disclosure further provides a robot control device, the device comprising:

[0031] The control module is used to control the robot to move to a second position corresponding to the first position where the box to be picked up is located, and then control the robot to obtain at least one image information of the shelf at the first position.

[0032] The processing module is used to control the movement of the robot's picking and placing device based on the image information, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through the ranging sensor on the picking and placing device.

[0033] The control module is also used to control the robot's picking and placing device to perform a picking operation on the box to be picked up if there is no obstacle.

[0034] Optionally, the processing module is specifically used to obtain the length information of the cargo box along the extension and retraction direction of the picking and placing device; obtain the field of view angle of the ranging sensor; and control the movement of the robot's picking and placing device based on the image information, length information and field of view angle.

[0035] Optionally, the processing module is specifically used to determine a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf based on the image information, the first surface being the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; and control the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle, and the length information.

[0036] Optionally, the processing module is specifically used to obtain position information of the ranging sensor; determine the height difference between the ranging sensor and the shelf based on the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; control the robot's picking and placing device to move based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface of the cargo box is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0037] Optionally, the processing module is specifically used to determine the target height difference based on the first distance, the field of view angle and the length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; and control the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0038] Optionally, the control module is also used to control the robot to move to a third position corresponding to the target storage location, and then control the robot to obtain image information of at least one shelf where the target storage location is located; based on the image information, determine whether there is an obstacle on the target storage location; if there is no obstacle, control the robot's picking and placing device to perform a placing operation on the target storage location.

[0039] Optionally, the image information is a two-dimensional image, and the control module is specifically used to control the robot's picking and placing device to be raised to a preset height, and then control the robot to obtain at least one image information of the shelf where the target storage location is located; based on whether the image information includes the identification information of the cargo box, determine whether there is an obstacle on the target storage location.

[0040] Optionally, the image information is a two-dimensional image, and the control module is specifically configured to input the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0041] Optionally, the image information is a three-dimensional image, and the control module is specifically used to analyze the image information to obtain the distribution of noise points in the image information; and determine whether there are obstacles on the target storage location based on the distribution of noise points.

[0042] Optionally, the control module is specifically configured to control the robot's cargo picking and placing device to rise and fall to the fourth position based on the acquired cargo box height information, and then control the robot to obtain at least one image information of the shelf at the first position.

[0043] In a fifth aspect, the present disclosure further provides a robot control device, the device comprising:

[0044] The acquisition module is used to respond to the first control instruction sent by the control device, move to the second position corresponding to the first position where the box to be picked up is located, and obtain at least one image information of the shelf at the first position through the image acquisition unit.

[0045] The sending module is used to send image information to the control device.

[0046] The control module is used to control the movement of the picking and placing device in response to the second control instruction sent by the control device, so that the control device detects whether there is an obstacle in the extension and retraction direction of the picking and placing device through the ranging sensor on the picking and placing device.

[0047] The control module is also used to control the picking and placing device to perform a picking operation on the cargo box to be picked up in response to a third control instruction sent by the control device. The third control instruction is sent by the control device when it detects that there is no obstacle in the extension and retraction direction of the picking and placing device.

[0048] Optionally, the control module is also used to receive a fourth control instruction sent by the control device, where the fourth control instruction is generated and sent by the control device based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle of the ranging sensor, and the length information of the cargo box along the telescopic direction of the picking and placing device. The first distance is the distance between the ranging sensor and the plane where the first surface of the cargo box is located; according to the fourth control instruction, the picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection point of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0049] Optionally, the acquisition module is specifically used to respond to the fifth control instruction sent by the control device, control the cargo picking and placing device to rise and fall to the fourth position, and obtain at least one image information of the shelf at the first position through the image acquisition unit. The fifth control instruction is generated and sent by the control device based on the acquired height information of the cargo box.

[0050] In a sixth aspect, the present disclosure further provides a robot control device, the device comprising:

[0051] The control module is used to control the robot to move to a third position corresponding to the target storage location, and then control the robot to obtain at least one image information of the shelf where the target storage location is located.

[0052] The processing module is used to determine whether there are obstacles at the target storage location based on the image information.

[0053] The control module is also used to control the robot's picking and placing device to perform a placing operation on the target storage location if there is no obstacle.

[0054] Optionally, the image information is a two-dimensional image, and the control module is specifically used to control the robot's picking and placing device to be raised to a preset height, and then control the robot to obtain at least one image information of a shelf where the target storage location is located.

[0055] The determination module is specifically used to determine whether there is an obstacle at the target storage location based on whether the image information includes the identification information of the cargo box.

[0056] Optionally, the image information is a two-dimensional image, and the control module is specifically configured to input the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0057] Optionally, the image information is a three-dimensional image, and the control module is specifically used to analyze the image information to obtain the distribution of noise points in the image information; and determine whether there are obstacles on the target storage location based on the distribution of noise points.

[0058] Optionally, the control module is further used to control the robot to move to a second position corresponding to the first position where the box to be picked up is located, and then control the robot to obtain image information of at least one shelf where the first position is located; according to the image information, control the robot's picking and placing device to move, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, control the robot's picking and placing device to perform a picking operation on the box to be picked up.

[0059] Optionally, the control module is specifically used to obtain the field of view angle of the ranging sensor; and control the robot's picking and placing device to move according to the image information, length information and field of view angle.

[0060] Optionally, the control module is specifically used to determine a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf based on the image information, the first surface being the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; and control the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and the length information.

[0061] Optionally, the control module is specifically used to obtain position information of the ranging sensor; determine the height difference between the ranging sensor and the shelf based on the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; control the robot's picking and placing device to move based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection, the target intersection is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface of the cargo box is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0062] Optionally, the control module is specifically used to determine the target height difference based on the first distance, the field of view angle and the length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; and control the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0063] Optionally, the control module is specifically configured to control the robot's cargo picking and placing device to rise and fall to the fourth position based on the acquired cargo box height information, and then acquire at least one image information of the shelf at the first position.

[0064] In the seventh aspect, the present disclosure also provides a control device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided in the first aspect and the optional manner of the first aspect, or the third aspect and the optional manner of the third aspect.

[0065] In an eighth aspect, the present disclosure also provides a robot comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided in the second aspect and the optional manner of the second aspect.

[0066] In a ninth aspect, the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method provided by the first aspect or an optional method of the first aspect, or the second aspect or an optional method of the second aspect, or the third aspect or an optional method of the third aspect.

[0067] In the tenth aspect, the present disclosure provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method provided by the first aspect or an optional manner of the first aspect, or the second aspect or an optional manner of the second aspect, or the third aspect or an optional manner of the third aspect.

[0068] The robot control method, device and equipment provided by the present disclosure control the robot to move to a second position corresponding to a first position where a box to be picked up is located, and then control the robot to obtain image information of at least one shelf where the first position is located; based on the image information, the robot's picking and placing device is controlled to move, so as to detect whether there is an obstacle in the extension and extension direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, the robot's picking and placing device is controlled to perform a picking operation on the box to be picked up, which can determine in real time whether there is an obstacle in the picking direction of the robot, and control the robot to pick up the goods according to the judgment result, thereby improving the safety of the robot and the success rate of picking up the goods. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of an application scenario of the robot control method provided by the present disclosure;

[0070] Figure 2 A schematic flow chart of a robot control method provided by the present disclosure;

[0071] Figure 3 A schematic diagram of the relative position relationship between the image acquisition unit and the cargo box provided by the present disclosure;

[0072] Figure 4 Another flowchart of the robot control method provided by the present disclosure;

[0073] Figure 5 A schematic diagram of the relative position relationship between the ranging sensor and the cargo box provided by the present disclosure;

[0074] Figure 6 Another schematic diagram of the relative position relationship between the ranging sensor and the cargo box provided by the present disclosure;

[0075] Figure 7 A schematic diagram of another flow chart of the robot control method provided by the present disclosure;

[0076] Figure 8 A schematic diagram of another flow chart of the robot control method provided by the present disclosure;

[0077] Figure 9 A schematic structural diagram of a robot control device provided by the present disclosure;

[0078] Figure 10Another schematic diagram of the structure of the control device of the robot provided by the present disclosure;

[0079] Figure 11 This is another schematic structural diagram of the control device of the robot provided by the present disclosure;

[0080] Figure 12 A schematic diagram of the structure of the control device provided by the present disclosure;

[0081] Figure 13 A schematic structural diagram of the robot provided by the present invention.

[0082] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0083] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0084] In existing technologies, the dispatching system controls robots to pick up or place goods based on stored bin or storage location information. However, because the dispatching system cannot obtain real-time details of the goods on the shelves and cannot determine whether there are obstacles in the robot's operating area on the shelves, interference may occur when the robot is picking up or placing goods, preventing the robot from completing its task successfully and even threatening its safety.

[0085] To solve the above problems, the present disclosure provides a robot control method. Before controlling the robot to pick up goods, the robot first collects corresponding data and determines whether there is an obstacle area in the target area around the box to be picked up. If there is no obstacle, the robot is controlled to obtain the box, thereby making up for the defect that the scheduling system cannot perceive in real time whether there are obstacles in the robot's operating area on the shelf, thereby improving the success rate of the robot's picking up goods and the safety of the robot.

[0086] Figure 1 A schematic diagram of an application scenario of the robot control method provided by the present disclosure, such as Figure 1 As shown, the method is applied to a scenario including a control device 11 , a robot 12 and a shelf 13 .

[0087] The control device 11 is in communication with the robot 12. The robot 12 includes an image acquisition unit, a cargo picking and placing device, and a distance measuring sensor, which is provided on the cargo picking and placing device.

[0088] The control device 11 is used to control the robot 12 to move to a second position corresponding to the first position where the box to be picked up is located, control the robot 12 to obtain at least one image information of the shelf where the first position is located, and control the picking and placing device of the robot 12 to move based on the image information, so as to detect whether there is an obstacle in the extension and extension direction of the picking and placing device through the ranging sensor on the picking and placing device; if there is no obstacle, control the picking and placing device of the robot 12 to perform a picking operation on the box to be picked up.

[0089] The first position where the box to be picked up is located is on shelf 13.

[0090] Figure 2 A schematic flow chart of a robot control method provided by the present disclosure, which is applied to control equipment such as Figure 2 As shown, the method includes:

[0091] S201: After controlling the robot to move to a second position corresponding to the first position where the box to be picked up is located, the robot is controlled to obtain at least one image information of the shelf at the first position.

[0092] The first position of the to-be-picked box is the position of the to-be-picked box on the shelf, such as the storage location of the to-be-picked box.

[0093] The robot is controlled to move to a second position corresponding to the first position of the to-be-collected container, that is, the robot's pick-up and placement device is controlled to face the to-be-collected container directly. Specifically, the robot may be controlled to move according to a dispatch instruction based on a ground code, until it reaches the ground code corresponding to the first position of the to-be-collected container. The robot's pick-up and placement device is then controlled to rise or fall to a height corresponding to the first position of the to-be-collected container. The robot's pick-up and placement device is then controlled to rotate a certain angle so that the robot's pick-up and placement device faces the storage location of the to-be-collected container directly.

[0094] Controlling the robot to obtain at least one image of the shelf at the first location may involve controlling an image acquisition unit on the robot to obtain at least one image of the shelf at the first location. Exemplarily, the image acquisition unit is a 3D camera, a 2D camera, or the like; accordingly, the image information may be 3D image information, such as a 3D point cloud, or 2D image information.

[0095] When the robot's image acquisition unit is set on its picking and placing device, the relative position of the picking and placing device and the cargo box can be adjusted to indirectly adjust the relative position of the image acquisition unit and the cargo box, so that the image acquisition unit can collect effective image information.

[0096] In one possible implementation, the robot is controlled to obtain at least one image information of the shelf at the first position through the image acquisition unit, including: according to the obtained height information of the cargo box, the robot's picking and placing device is controlled to rise and fall to the fourth position, and then the image information of the shelf at the first position is obtained.

[0097] The control device may obtain the height information of the cargo box by reading the pre-stored height information of the cargo box from a virtual or physical storage medium; or it may receive the height information of the cargo box sent by an upper system, such as a scheduling system; or it may obtain the height information of the cargo box by controlling the robot to obtain image information of at least one shelf and then analyzing the obtained image information.

[0098] The fourth position may be any position corresponding to the height of the cargo box. For example, when the robot's cargo picking and placing device is located at the fourth position, the robot's image acquisition unit maintains a consistent height with the center of the cargo box.

[0099] Through this method, the robot's picking and placing device can be controlled to quickly pre-adjust to the fourth position, so that the image information subsequently acquired by the robot's image acquisition unit includes cargo box information, avoiding the acquisition of invalid image information. At the same time, when the ranging sensor of the picking and placing device detects obstacles, it is possible to avoid misjudgment caused by the shelf layer. Through rapid coarse adjustment, the foundation is laid for the subsequent fine adjustment of the position of the robot's picking and placing device.

[0100] like Figure 3 A schematic diagram of the relative position relationship between the image acquisition unit and the cargo box provided by the present disclosure, wherein the cargo box 32 is located on the shelf 33, and the image acquisition unit 311 is provided on the cargo pick-up and placement device 31. Figure 3 In the relative positional relationship between the image acquisition unit and the cargo box shown, the image information captured by image acquisition unit 311 does not include information about cargo box 32, making it unusable for subsequent fine-tuning of the robot's pick-up and placement mechanism. This image is considered invalid. To ensure that image acquisition unit 311 captures a valid image, the pick-up and placement mechanism should be lowered a certain distance to allow image acquisition unit 311 to capture cargo box information. To avoid this repeated adjustment, a more reasonable target position can be determined based on the acquired cargo box height information, and the robot's pick-up and placement mechanism can be controlled to rise and fall to this target position, allowing the robot's image acquisition device to capture cargo box information.

[0101] S202: Control the robot's picking and placing device to move according to the image information, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a distance measuring sensor on the picking and placing device.

[0102] The robot's cargo picking and placing device may be a cargo fork, and the distance measuring sensor may be arranged on the tines of the cargo fork.

[0103] By analyzing the image information, if it is determined that the position of the robot's picking and placing device can meet the requirements, and the ranging sensor on the picking and placing device can be used to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device, the control device can control the robot's picking and placing device to move 0 meters; if it is determined that the position of the robot's picking and placing device cannot meet the requirements, the control device can control the robot's picking and placing device to move in any form, such as forward and backward, up and down, so that the control device can detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through the ranging sensor on the picking and placing device.

[0104] Specifically, by analyzing and identifying image information, separating the shelves and cargo boxes, and further determining the position, height and other related information of the shipping boxes, the robot's picking and placing device is controlled to move so that while the robot is aiming at the cargo boxes to be picked up, the ranging sensors on its picking and placing device avoid the shelves and cargo boxes, and determine whether there are obstacles in the extension and retraction direction of the picking and placing device through the signals detected by the ranging sensors.

[0105] The distance measuring sensor can be any one of the following sensors: a photoelectric sensor, a laser sensor, or an ultrasonic sensor.

[0106] S203: If there is no obstacle, the picking and placing device of the robot is controlled to perform a picking operation on the box to be picked up.

[0107] If it is determined that there is no obstacle in the extension and retraction direction of the pick-up and place device, it means that no collision will occur when the robot extends and retracts its pick-up and place device to pick up the cargo box. Therefore, the pick-up and place device of the robot can be controlled to perform the picking operation on the cargo box to be picked up.

[0108] The control method of the robot provided by the present disclosure controls the robot to move to a second position corresponding to a first position where a box to be picked up is located, and then controls the robot to obtain image information of at least one shelf where the first position is located; based on the image information, controls the robot's picking and placing device to move, so as to detect whether there is an obstacle in the extension and extension direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, controls the robot's picking and placing device to perform a picking operation on the box to be picked up, and can accurately identify whether there are obstacles around the box to be picked up through the ranging sensor on the robot's picking and placing device, and only when there are no obstacles will the picking operation be performed on the box to be picked up, thereby improving the robot's success rate in picking up goods and improving the robot's safety and reliability.

[0109] Figure 4 Another flow chart of the robot control method provided by the present disclosure is provided, and the method is applied to control equipment such as Figure 4 As shown, the method includes:

[0110] S401: After controlling the robot to move to a second position corresponding to the first position where the box to be picked up is located, the robot is controlled to obtain at least one image information of the shelf at the first position.

[0111] S401 and S201 have the same or corresponding technical features. For detailed description, please refer to S201 and will not be repeated here.

[0112] S402: Obtain the length information of the cargo box along the extension and contraction direction of the cargo picking and placing device.

[0113] The control device can obtain the length information of the cargo box along the extension and retraction direction of the cargo picking and placing device from other electronic devices; or obtain the length information of the cargo box along the extension and retraction direction of the cargo picking and placing device from a storage medium. The length information can be stored in the storage medium by the control device when the cargo box is put into storage.

[0114] For example, the length information of the cargo box along the extension and retraction direction of the cargo pick-up and placement device may be issued by the scheduling system, or may be a pre-stored default value, which may be greater than or equal to the length of all cargo boxes.

[0115] S403: Obtain the field of view angle of the ranging sensor.

[0116] The field of view angle of the ranging sensor is related to the model of the ranging sensor, and the pre-stored field of view angle of the ranging sensor can be obtained.

[0117] S404: Control the robot's pick-up and placement device to move according to the image information, length information, and field of view angle, so as to detect whether there is an obstacle in the extension and retraction direction of the pick-up and placement device through a distance measuring sensor on the pick-up and placement device.

[0118] When the pick-and-place mechanism reaches a box, it only needs to extend slightly longer than the box's length along the mechanism's extension direction. Therefore, the ranging sensor should be able to detect an area slightly longer than the box's length and equal to the mechanism's width along the mechanism's extension direction. To avoid false detections, the robot's pick-and-place mechanism must be controlled to move based on image information, length information, and field of view, ensuring that the shelf is not within the ranging sensor's detection area.

[0119] In one possible implementation, the robot's picking and placing device is controlled to move based on image information, length information, and field of view angle, including: determining a first distance between a ranging sensor and a plane on which a first surface of a cargo box is located, and a height difference between an image acquisition unit of the robot and a shelf based on the image information, the first surface being a surface of the cargo box opposite to and close to the ranging sensor of the robot; and controlling the robot's picking and placing device to move based on the first distance, the height difference between the image acquisition unit of the robot and the shelf, the field of view angle, and the length information.

[0120] Specifically, when a shelf code is affixed to the shelf, the first distance between the ranging sensor and the plane where the first surface of the cargo box is located is determined based on the image information. This can be done by scanning the shelf code with a two-dimensional camera to obtain image information, and then determining the first distance by identifying and parsing the shelf code. When there is no shelf code on the shelf, the first distance between the ranging sensor and the plane where the first surface of the cargo box is located is determined based on the image information. This can be done by obtaining a point cloud image with a three-dimensional camera and determining the first distance by analyzing the point cloud image.

[0121] Optionally, the height difference between the robot's image acquisition unit and the shelf can be obtained by identifying and parsing the shelf code.

[0122] This method can more accurately control the movement of the robot's picking and placing device based on the relative positions of the ranging sensors and the shelves and cargo boxes, and detect whether there are obstacles in the extension and retraction direction of the picking and placing device through the ranging sensors on the picking and placing device.

[0123] Furthermore, in a possible implementation, controlling the robot's picking and placing device to move according to the first distance, height difference, field of view angle, and length information includes: acquiring position information of the ranging sensor; determining the height difference between the ranging sensor and the shelf according to the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; controlling the robot's picking and placing device to move according to the first distance, the height difference between the ranging sensor and the shelf, the field of view angle, and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference being the height difference between the ranging sensor and the target intersection point, the target intersection point being the intersection point of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, the second surface of the cargo box being the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0124] Among them, after obtaining the height difference between the robot's image acquisition unit and the shelf by decoding the shelf code, the height difference between the ranging sensor and the image acquisition unit can be obtained through the stored configuration information, and then the height difference between the ranging sensor and the shelf can be converted.

[0125] Figure 5 This is a schematic diagram of the relative positional relationship between the ranging sensor and the cargo box provided by the present disclosure. The ranging sensor 312 is provided on the robot's pick-and-place device 31, and the cargo box 32 is located on a shelf 33. d1 is the first distance between the ranging sensor and the plane containing the first surface of the cargo box; d2 is the length of the cargo box along the extension and retraction direction of the pick-and-place device; a is the field of view of the ranging sensor; p1 is the intersection of the lower edge of the ranging sensor's field of view with the plane containing the second surface of the cargo box; d4 is the height difference between the ranging sensor and the shelf; and d3 is the target height difference, i.e., the height difference between the ranging sensor and the intersection of the lower edge of the field of view with the plane containing the second surface of the cargo box. Based on the first distance, the height difference between the ranging sensor and the shelf, the field of view, and the length information, the robot's pick-and-place device is controlled to move so that d3 = tan(a / 2)*(d1+d2), and d3 ≤ d4.

[0126] Optionally, considering that there may be certain errors in actual applications, a certain error margin may be set so that the target height difference satisfies d3+Δd≤d4, where Δd1 is the allowable error margin.

[0127] This method can ensure that the shelf is not included in the detection area of ​​the ranging sensor, improve the accuracy of the detection results, and thus increase the success rate of the robot's picking up of goods and ensure the safety of the robot.

[0128] In another possible implementation, determining a first distance between a ranging sensor and a plane on which a first surface of a cargo box is located based on image information includes: determining a position of a first surface of a shelf based on the image information, the first surface being a surface of the cargo box that is opposite to the ranging sensor of the robot and close to the ranging sensor; obtaining a position of the ranging sensor; and determining a first distance between the position of the ranging sensor and the plane on which the first surface is located.

[0129] The method determines the shelf in the image by analyzing image information, further determines the position of the first surface of the shipping box, and accurately determines the distance between the ranging sensor and the plane where the first surface of the shipping box is located based on the position information of the ranging sensor included in the pre-stored configuration information.

[0130] Optionally, in order to improve the accuracy of the detection results, when selecting the ranging sensor, the following condition must be met: 2tan(a / 2)*d1>=d5+△d2, where d1 is the first distance between the ranging sensor and the plane where the first surface of the cargo box is located, d5 is the width of the cargo pick-up and release device, and △d2 is the allowable error margin.

[0131] Figure 6 Another schematic diagram of the relative positional relationship between the ranging sensor and the cargo box provided by the present disclosure, wherein the ranging sensor 311 is arranged on the robot's pick-up and placement device 31, and the cargo box 32 is located on the shelf 33. d1 is the first distance between the ranging sensor and the plane where the first surface of the cargo box is located; a is the field of view of the ranging sensor; d5 is the width of the pick-up and placement device, and △d2 is the allowable error margin.

[0132] S405: If there is no obstacle, the picking and placing device of the robot is controlled to perform a picking operation on the box to be picked up.

[0133] S405 has the same or corresponding technical features as S203. For detailed description, please refer to S203 and will not be repeated here.

[0134] Optionally, if there is an obstacle, a notification message is sent to inform that the container is not accessible.

[0135] For example, if there is an obstacle, a notification message is sent to the dispatch system to report that the dispatch system is abnormal and the box cannot be taken.

[0136] Optionally, the method also includes: after controlling the robot to move to a third position corresponding to the target storage location, controlling the robot to obtain at least one image information of the shelf where the target storage location is located; based on the image information, determining whether there is an obstacle on the target storage location, and if there is no obstacle, controlling the robot's picking and placing device to perform a placing operation on the target storage location.

[0137] For example, after controlling the robot to move to the ground code position corresponding to the scheduling instruction, the robot's picking and placing device is controlled to rise and fall to the floor height corresponding to the scheduling instruction, and then the robot's picking and placing device is controlled to rotate a certain angle, such as 90 degrees, so that the picking and placing device is directly facing the storage location.

[0138] The image information may be two-dimensional image information or three-dimensional image information.

[0139] If the image information is a two-dimensional image, in one possible implementation, the robot is controlled to obtain image information of at least one shelf where the target storage location is located, including: after controlling the robot's picking and placing device to be raised to a preset height, the robot is controlled to obtain image information of at least one shelf where the target storage location is located.

[0140] Accordingly, determining whether there is an obstacle at the target storage location based on the image information includes: determining whether there is an obstacle at the target storage location based on whether the image information includes identification information of the cargo box.

[0141] The preset height may be determined based on the height of the labels on general material boxes.

[0142] Exemplarily, the robot's picking and placing device is controlled to rise to a certain height so that the robot's camera is aimed at the coding height of a general material box, and the robot is controlled to obtain at least one image information of the shelf where the target storage location is located. By parsing the image information, it is determined whether the image information includes the identification information of the cargo box. If the identification information of the cargo box is included, it is determined that there is an obstacle on the target storage location, that is, a cargo box is placed; if the identification information of the cargo box is not included, it is determined that there is no obstacle on the target storage location, that is, the storage location is an empty storage location.

[0143] This method can quickly determine whether there is a cargo box at the target storage location by collecting the corresponding two-dimensional image of the shelf when the shelf is affixed with a shelf code.

[0144] If the image information is a two-dimensional image, in another possible implementation, determining whether there is an obstacle at the target storage location based on the image information includes: inputting the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0145] For example, multiple images are collected in advance under different working conditions, environments, lighting conditions, camera exposures, and scenarios involving the presence or absence of containers at the storage location. These images are calibrated as sample data and trained through deep learning to develop a judgment model. Before the robot releases the goods, the image information corresponding to the current storage location captured by the robot is input into the pre-trained judgment model to determine whether there are obstacles at the target storage location.

[0146] This method is combined with a training model to improve judgment efficiency and accuracy.

[0147] The image information is a three-dimensional image. In one possible implementation, determining whether there is an obstacle at the target storage location based on the image information includes: parsing the image information to obtain the distribution of noise points in the image information; and determining whether there is an obstacle at the target storage location based on the noise point distribution.

[0148] For example, based on the distribution of noise points, it is determined whether there are large areas of noise points within the preset field of view. If there are large areas of noise points, it is determined that a container is present within the preset field of view; if there are no large areas of noise points, it is determined that no container is present within the preset field of view. The preset field of view can be an area above the shelf level where the target storage location is located and below the upper shelf board of the target storage location within a preset depth and width range, which includes the target storage location. The preset depth is less than or equal to the sum of the camera working distance and the length of the container. The preset width can be the largest container that the pick-and-place device can access.

[0149] This method can quickly determine whether there is a cargo box at the target storage location by collecting and analyzing three-dimensional images even when the shelf does not have a shelf code.

[0150] The robot control method provided by the present disclosure, based on the above-mentioned embodiments, further obtains the length information of the cargo box along the extension and retraction direction of the cargo picking and placing device; obtains the field of view angle of the ranging sensor; and controls the movement of the robot's cargo picking and placing device according to the image information, length information and field of view. This can reasonably adjust the position of the robot's cargo picking and placing device, improve the accuracy of the detection results, and further improve the safety of the robot and the success rate of cargo picking.

[0151] Figure 7 This is another flow chart of the control method of the robot provided by the present disclosure, which is applied to the robot. The robot includes a picking and placing device, an image acquisition unit, and a ranging sensor. The ranging sensor is arranged on the picking and placing device, such as Figure 7 As shown, the method includes:

[0152] S701. In response to a first control instruction sent by a control device, after moving to a second position corresponding to a first position where a box to be picked up is located, obtain at least one image information of the shelf at the first position through an image acquisition unit.

[0153] S702: Send image information to the control device.

[0154] S703: In response to the second control instruction sent by the control device, control the cargo picking and placing device to move, so that the control device detects whether there is an obstacle in the extension and retraction direction of the cargo picking and placing device through a distance measuring sensor on the cargo picking and placing device.

[0155] S704: In response to a third control instruction sent by the control device, control the picking and placing device to perform a picking operation on the box to be picked up. The third control instruction is sent by the control device when it detects that there is no obstacle in the extension and retraction direction of the picking and placing device.

[0156] Optionally, it also includes: receiving a fourth control instruction sent by the control device, the fourth control instruction is generated and sent by the control device based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle of the ranging sensor and the length information of the cargo box along the telescopic direction of the picking and placing device, the first distance is the distance between the ranging sensor and the plane where the first surface of the cargo box is located; according to the fourth control instruction, the picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection, the target intersection is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0157] Optionally, it also includes: in response to a fifth control instruction sent by the control device, controlling the cargo picking and placing device to rise and fall to the fourth position, obtaining at least one image information of the shelf at the first position through the image acquisition unit, and the fifth control instruction is generated and sent by the control device based on the acquired height information of the cargo box.

[0158] The robot control method provided by the present disclosure sends image information to a control device; controls the robot's picking and placing device to move in response to a second control instruction sent by the control device, so that the control device detects whether there is an obstacle in the extension and retraction direction of the picking and placing device through a ranging sensor on the picking and placing device; and controls the picking and placing device to perform a picking operation on a to-be-picked box in response to a third control instruction sent by the control device when it is detected that there is no obstacle in the extension and retraction direction of the picking and placing device. This can improve the safety of the robot and the success rate of picking and placing.

[0159] Figure 8 This is another flow chart of a robot control method provided by the present disclosure, which is applied to a control device. The method includes:

[0160] S801: After controlling the robot to move to a third position corresponding to a target storage location, the robot is controlled to obtain at least one image information of a shelf where the target storage location is located.

[0161] S802: Determine whether there are obstacles at the target storage location based on the image information.

[0162] The image information is a two-dimensional image. In one possible implementation, the robot is controlled to obtain image information of at least one shelf where the target storage location is located, including: after the picking and placing device of the robot is controlled to be raised to a preset height, the robot is controlled to obtain image information of at least one shelf where the target storage location is located.

[0163] Accordingly, determining whether there is an obstacle at the target storage location based on the image information includes: determining whether there is an obstacle at the target storage location based on whether the image information includes identification information of the cargo box.

[0164] The image information is a two-dimensional image. In another possible implementation, determining whether there is an obstacle at the target storage location based on the image information includes: inputting the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0165] The image information is a three-dimensional image. In one possible implementation, determining whether there is an obstacle at the target storage location based on the image information includes: parsing the image information to obtain the distribution of noise points in the image information; and determining whether there is an obstacle at the target storage location based on the noise point distribution.

[0166] S803: If there is no obstacle, control the robot's picking and placing device to perform a placing operation on the target storage location.

[0167] Optionally, the method further includes:

[0168] S804: After controlling the robot to move to a second position corresponding to the first position where the box to be picked up is located, the robot is controlled to obtain at least one image information of the shelf at the first position.

[0169] S805 . Control the robot's picking and placing device to move according to the image information, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a distance measuring sensor on the picking and placing device.

[0170] In one possible implementation, the robot's picking and placing device is controlled to move based on the image information, including: obtaining length information of the cargo box along the extension and retraction direction of the picking and placing device; obtaining the field of view angle of the ranging sensor; and controlling the robot's picking and placing device to move based on the image information, length information, and field of view angle.

[0171] Further, optionally, the robot's picking and placing device is controlled to move according to the image information, length information and field of view angle, including: determining a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf according to the image information, the first surface being the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; and controlling the robot's picking and placing device to move according to the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and the length information.

[0172] Further, optionally, controlling the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle and the length information includes: obtaining position information of the ranging sensor; determining the height difference between the ranging sensor and the shelf based on the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; controlling the robot's picking and placing device to move based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection point of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, the second surface being the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0173] Further, optionally, based on the first distance, field of view angle and length information, the robot's picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, including: determining the target height difference based on the first distance, field of view angle and length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; controlling the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0174] S806: If there is no obstacle, control the robot's picking and placing device to perform a picking operation on the box to be picked up.

[0175] The control method of the robot provided by the present disclosure controls the robot to move to a third position corresponding to a target storage location, and then controls the robot to obtain image information of at least one shelf where the target storage location is located; determines whether there is an obstacle on the target storage location based on the image information, thereby improving the success rate of the robot in placing goods; controls the robot to move to a second position corresponding to a first position where a box to be picked up is located, and then controls the robot to obtain image information of at least one shelf where the first position is located; controls the robot's picking and placing device to move based on the image information, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, controls the robot's picking and placing device to perform a picking operation on the box to be picked up, thereby improving the success rate of the robot in picking up goods and the safety of the robot.

[0176] Figure 9 A schematic diagram of the structure of the control device of the robot provided by the present disclosure is shown as follows: Figure 9 As shown, the device includes:

[0177] A control module 91 is configured to control the robot to move to a second position corresponding to the first position where the box to be picked up is located, and then control the robot to obtain at least one image of the shelf at the first position;

[0178] The processing module 92 is used to control the movement of the robot's picking and placing device according to the image information, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through the ranging sensor on the picking and placing device.

[0179] The control module 91 is also used to control the robot's picking and placing device to perform a picking operation on the box to be picked up if there is no obstacle.

[0180] Optionally, the processing module 92 is specifically used to obtain the length information of the cargo box along the extension and retraction direction of the picking and placing device; obtain the field of view angle of the ranging sensor; and control the movement of the robot's picking and placing device based on the image information, length information and field of view angle.

[0181] Optionally, the processing module 92 is specifically used to determine a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf based on the image information, the first surface being the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; and control the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle, and the length information.

[0182] Optionally, the processing module 92 is specifically used to obtain the position information of the ranging sensor; determine the height difference between the ranging sensor and the shelf according to the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; control the robot's picking and placing device to move according to the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection, the target intersection is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0183] Optionally, the processing module 92 is specifically used to determine the target height difference based on the first distance, the field of view angle and the length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; and control the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0184] Optionally, the control module 91 is also used to control the robot to move to a third position corresponding to the target storage location, and then control the robot to obtain image information of at least one shelf where the target storage location is located; based on the image information, determine whether there is an obstacle on the target storage location; if there is no obstacle, control the robot's picking and placing device to perform a placing operation on the target storage location.

[0185] Optionally, the image information is a two-dimensional image, and the control module 91 is specifically used to control the robot's picking and placing device to be raised to a preset height, and then control the robot to obtain image information of at least one shelf where the target storage location is located; and determine whether there is an obstacle on the target storage location based on whether the image information includes identification information of the cargo box.

[0186] Optionally, the image information is a two-dimensional image, and the control module 91 is specifically configured to input the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0187] Optionally, the image information is a three-dimensional image, and the control module 91 is specifically configured to analyze the image information to obtain a distribution of noise points in the image information; and determine whether there is an obstacle at the target storage location based on the distribution of noise points.

[0188] Optionally, the control module 91 is specifically configured to control the robot's cargo picking and placing device to rise and fall to the fourth position based on the acquired cargo box height information, and then control the robot to acquire at least one image information of the shelf at the first position.

[0189] The robot's control device can perform the above Figure 2 or Figure 4 The content and effects of the robot control method provided in the illustrated embodiment can be referred to in the method embodiment section, which will not be described in detail.

[0190] Figure 10 Another structural diagram of the control device of the robot provided by the present disclosure is as follows: Figure 10 As shown, the device includes:

[0191] The acquisition module 101 is used to respond to the first control instruction sent by the control device, move to the second position corresponding to the first position where the box to be picked up is located, and acquire at least one image information of the shelf at the first position through the image acquisition unit.

[0192] The sending module 102 is configured to send image information to a control device.

[0193] The control module 103 is used to control the robot's picking and placing device to move in response to the second control instruction sent by the control device, so that the control device detects whether there is an obstacle in the extension and retraction direction of the picking and placing device through the ranging sensor on the picking and placing device.

[0194] The control module 103 is further configured to control the picking and placing device to perform a picking operation on the box to be picked up in response to a third control instruction sent by the control device. The third control instruction is sent by the control device when it detects that there is no obstacle in the extension and retraction direction of the picking and placing device.

[0195] Optionally, the control module 103 is also used to receive a fourth control instruction sent by the control device, where the fourth control instruction is generated and sent by the control device based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle of the ranging sensor, and the length information of the cargo box along the telescopic direction of the picking and placing device. The first distance is the distance between the ranging sensor and the plane where the first surface of the cargo box is located; according to the fourth control instruction, the picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection point of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0196] Optionally, the acquisition module 101 is specifically used to respond to a fifth control instruction sent by the control device, control the cargo picking and placing device to rise and fall to the fourth position, and obtain at least one image information of the shelf at the first position through the image acquisition unit. The fifth control instruction is generated and sent by the control device based on the acquired height information of the cargo box.

[0197] The robot's control device can perform the above Figure 7The content and effects of the robot control method provided in the illustrated embodiment can be referred to in the method embodiment section, which will not be described in detail.

[0198] Figure 11 This is another structural diagram of the control device of the robot provided by the present disclosure, such as Figure 11 As shown, the device includes:

[0199] The control module 111 is configured to control the robot to move to a third position corresponding to the target storage location, and then control the robot to obtain at least one image information of a shelf where the target storage location is located.

[0200] The processing module 112 is used to determine whether there is an obstacle at the target storage location based on the image information.

[0201] The control module 111 is further configured to control the robot's picking and placing device to perform a placing operation on the target storage location if no obstacle exists.

[0202] Optionally, the image information is a two-dimensional image, and the control module 111 is specifically configured to control the robot's picking and placing device to be raised to a preset height, and then control the robot to obtain at least one image information of a shelf where the target storage location is located.

[0203] The determination module 112 is specifically configured to determine whether there is an obstacle at the target storage location based on whether the image information includes identification information of the cargo box.

[0204] Optionally, the image information is a two-dimensional image, and the control module 111 is specifically configured to input the image information into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

[0205] Optionally, the image information is a three-dimensional image, and the control module 111 is specifically configured to analyze the image information to obtain a distribution of noise points in the image information; and determine whether there is an obstacle at the target storage location based on the distribution of noise points.

[0206] Optionally, the control module 111 is further configured to control the robot to move to a second position corresponding to the first position where the box to be picked up is located, and then control the robot to obtain at least one image information of the shelf at the first position; based on the image information, control the robot's picking and placing device to move, so as to detect whether there is an obstacle in the extension and retraction direction of the picking and placing device through a ranging sensor on the picking and placing device; if there is no obstacle, control the robot's picking and placing device to perform a picking operation on the box to be picked up.

[0207] Optionally, the control module 111 is specifically configured to obtain the field of view angle of the ranging sensor; and control the robot's picking and placing device to move according to the image information, length information, and field of view angle.

[0208] Optionally, the control module 111 is specifically used to determine a first distance between the ranging sensor and the plane where the first surface of the cargo box is located, and a height difference between the robot's image acquisition unit and the shelf based on the image information, where the first surface is the surface of the cargo box opposite to the robot's ranging sensor and close to the ranging sensor; and control the robot's picking and placing device to move based on the first distance, the height difference between the robot's image acquisition unit and the shelf, the field of view angle, and the length information.

[0209] Optionally, the control module 111 is specifically used to obtain position information of the ranging sensor; determine the height difference between the ranging sensor and the shelf based on the height difference between the robot's image acquisition unit and the shelf and the position information of the ranging sensor; control the robot's picking and placing device to move based on the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection, the target intersection is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface of the cargo box is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

[0210] Optionally, the control module 111 is specifically used to determine the target height difference based on the first distance, the field of view angle and the length information: d3 = tan(a / 2)*(d1+d2), where a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; and control the robot's picking and placing device to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

[0211] The robot's control device can perform the above Figure 8 The content and effects of the robot control method provided in the illustrated embodiment can be referred to in the method embodiment section, which will not be described in detail.

[0212] Figure 12 A schematic diagram of the structure of the control device provided by the present disclosure, such as Figure 12 As shown, the control device of this embodiment includes: a processor 121 and a memory 122; the processor 121 is communicatively connected to the memory 122. The memory 122 is used to store computer programs. The processor 121 is used to call the computer program stored in the memory 122 to implement the method of the above embodiment.

[0213] Optionally, the control device further includes: a transceiver 123, configured to communicate with other devices.

[0214] The control device can perform the above Figure 2 、 Figure 4 or Figure 8The content and effects of the robot control method provided in the illustrated embodiment can be referred to in the method embodiment section, which will not be described in detail.

[0215] Figure 13 A schematic diagram of the structure of the robot provided by the present disclosure is shown in FIG. Figure 13 As shown, the robot of this embodiment includes: a processor 131 and a memory 132; the processor 131 is communicatively connected to the memory 132. The memory 132 is used to store computer programs. The processor 131 is used to call the computer program stored in the memory 132 to implement the method of the above embodiment.

[0216] Optionally, the robot further includes a transceiver 133 for communicating with other devices.

[0217] The control device can perform the above Figure 7 The content and effects of the robot control method provided in the illustrated embodiment can be referred to in the method embodiment section, which will not be described in detail.

[0218] The present disclosure further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement a method as described in any of the above method embodiments.

[0219] The computer-executable instructions stored in the computer-readable storage medium can implement the above-mentioned robot control method when executed by the processor. Its content and effects can be referred to the method embodiment part, which will not be described in detail.

[0220] The present disclosure also provides a computer program product, including a computer program / instruction, which implements the method in any of the above method embodiments when executed by a processor.

[0221] The computer-executable instructions stored in the computer-readable storage medium can implement the above-mentioned robot control method when executed by the processor. Its content and effects can be referred to the method embodiment part, which will not be described in detail.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A robot control method, characterized in that: Applied to a control device, the method includes: After controlling the robot to move to a third position corresponding to the target storage location, controlling the robot to obtain first image information of at least one shelf where the target storage location is located; Determining whether there is an obstacle at the target storage location based on the first image information; If there is no obstacle, the robot's picking and placing device is controlled to perform a placing operation on the target storage location; the picking and placing device is a fork; The method further comprises: After controlling the robot to move to a second position corresponding to the first position where the box to be picked up is located, controlling the robot to obtain at least one second image information of the shelf where the first position is located; Obtaining length information of the cargo box along the extension and retraction direction of the cargo picking and placing device; Obtaining the field of view angle of the ranging sensor on the cargo picking and placing device; determining, based on the second image information, a first distance between the ranging sensor and a plane on which a first surface of the cargo box is located, and a height difference between the image acquisition unit of the robot and the shelf, where the first surface is a surface of the cargo box that faces the ranging sensor of the robot and is close to the ranging sensor; Controlling the robot's picking and placing device to move according to the first distance, the height difference between the image acquisition unit and the shelf, the length information, and the field of view angle, so as to detect, by the ranging sensor, whether there is an obstacle in the extension and retraction direction of the picking and placing device; If there is no obstacle, the picking and placing device of the robot is controlled to perform a picking operation on the box to be picked up.

2. The method according to claim 1, characterized in that The first image information is a two-dimensional image, The controlling the robot to obtain first image information of at least one shelf where the target storage location is located includes: After controlling the picking and placing device of the robot to be raised to a preset height, controlling the robot to obtain first image information of at least one shelf where the target storage location is located; Accordingly, determining whether there is an obstacle at the target storage location based on the first image information includes: Whether there is an obstacle at the target storage location is determined based on whether the first image information includes identification information of the cargo box.

3. The method according to claim 1, characterized in that The first image information is a two-dimensional image, The determining, based on the first image information, whether there is an obstacle at the target storage location includes: The first image information is input into a pre-trained judgment model to determine whether there is an obstacle at the target storage location.

4. The method according to claim 1, wherein The first image information is a three-dimensional image, The determining, based on the first image information, whether there is an obstacle at the target storage location includes: Analyzing the first image information to obtain noise distribution in the first image information; Determine whether there is an obstacle on the target storage location based on the noise point distribution.

5. The method according to claim 4, characterized in that The controlling the robot's picking and placing device to move according to the first distance, the height difference between the image acquisition unit and the shelf, the field of view angle, and the length information includes: Obtaining location information of the ranging sensor; determining a height difference between the ranging sensor and the shelf based on the height difference between the image acquisition unit of the robot and the shelf and position information of the ranging sensor; According to the first distance, the height difference between the ranging sensor and the shelf, the field of view angle and the length information, the robot's picking and placing device is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf, the target height difference is the height difference between the ranging sensor and the target intersection point, the target intersection point is the intersection of the lower edge of the field of view angle and the plane where the second surface of the cargo box is located, and the second surface of the cargo box is the surface of the cargo box opposite to the ranging sensor of the robot and away from the ranging sensor.

6. The method according to claim 5, characterized in that The controlling the robot's picking and placing device to move according to the first distance, the height difference between the ranging sensor and the shelf, the field of view angle, and the length information so that a target height difference is less than or equal to the height difference between the ranging sensor and the shelf includes: Determine the target height difference according to the first distance, the field of view angle, and the length information: d3=tan(a / 2) (d1+ d2) Wherein, a is the field of view angle, d1 is the first distance, d2 is the length information, and d3 is the target height difference; The picking and placing device of the robot is controlled to move so that the target height difference is less than or equal to the height difference between the ranging sensor and the shelf.

7. The method according to any one of claims 1 to 6, characterized in that The controlling the robot to obtain second image information of at least one shelf where the first position is located includes: According to the acquired height information of the cargo box, after controlling the cargo picking and placing device of the robot to rise and fall to the fourth position, the robot is controlled to obtain at least one second image information of the shelf at the first position.

8. A robot control device, characterized in that: include: a control module, configured to control the robot to move to a third position corresponding to a target storage location, and then control the robot to obtain first image information of at least one shelf where the target storage location is located; a processing module, configured to determine whether there is an obstacle at the target storage location based on the first image information; The control module is further configured to control the robot's picking and placing device to perform a placing operation on the target storage location if no obstacle exists; the picking and placing device is a fork; and After controlling the robot to move to a second position corresponding to the first position where the box to be picked up is located, controlling the robot to obtain at least one second image information of the shelf where the first position is located; Obtaining length information of the cargo box along the extension and retraction direction of the cargo picking and placing device; Obtaining the field of view angle of the ranging sensor on the cargo picking and placing device; determining, based on the second image information, a first distance between the ranging sensor and a plane on which a first surface of the cargo box is located, and a height difference between the image acquisition unit of the robot and the shelf, where the first surface is a surface of the cargo box that faces the ranging sensor of the robot and is close to the ranging sensor; Controlling the robot's picking and placing device to move according to the first distance, the height difference between the image acquisition unit and the shelf, the length information, and the field of view angle, so as to detect, by the ranging sensor, whether there is an obstacle in the extension and retraction direction of the picking and placing device; If there is no obstacle, the picking and placing device of the robot is controlled to perform a picking operation on the box to be picked up.

9. A robot, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and when a processor executes the computer program or instructions, the method according to any one of claims 1 to 7 is implemented.

Images