Control method, device, equipment and readable storage medium of warehouse robot

Abstract

The present invention provides a control method, device, equipment, and readable storage medium for a warehouse robot. The method of the present invention, before executing a handling task, uses an image acquisition device to capture image data of a target storage location. Based on the image data of the target storage location, it is determined whether execution conditions for the handling task are currently met. If it is determined that the execution conditions for the handling task are met, that is, if the handling device can execute the handling task without danger, the handling device is controlled to execute the handling task, thereby avoiding danger and improving the safety of the warehouse robot.

Description

[0001] This application is a divisional application of the invention patent application with application number 202010537646.9 submitted to the China Patent Office, application date June 12, 2020, and invention name “Control method, device, equipment and readable storage medium for warehousing robot”. Technical Field

[0002] The present invention relates to the field of intelligent warehousing technology, and in particular to a control method, device, equipment and readable storage medium for a warehousing robot. Background Art

[0003] With the networking and intelligence of smart manufacturing and warehousing logistics, warehousing logistics plays a very important role in the production and management process of enterprises. In the field of smart warehousing, it is becoming more and more common for warehousing robots to replace workers in handling goods.

[0004] In existing intelligent warehousing systems, shelf vibrations and human error can cause bins to shift within storage or fall from shelves. Warehouse robots can collide with these bins when retrieving them or passing through them, creating safety risks for warehouse robots when storing and retrieving bins. Summary of the Invention

[0005] The present invention provides a control method, device, equipment and readable storage medium for a warehouse robot, which are used to solve the problem of low safety of the warehouse robot.

[0006] One aspect of the present invention is to provide a control method for a warehouse robot, wherein the warehouse robot has a handling device and an image acquisition device, comprising:

[0007] The image acquisition device acquires image data of a target storage location corresponding to a handling task; based on the image data of the target storage location, if it is determined that the execution conditions of the handling task are met, the handling device is controlled to execute the handling task.

[0008] In a possible implementation, the collecting of image data of a target storage location corresponding to a handling task by the image collection device includes:

[0009] When the warehouse robot moves to the target position corresponding to the target storage location, the image acquisition device is controlled to start and collect image data of the target storage location; or, when the warehouse robot moves to a preset range around the target storage location, the image acquisition device is controlled to start and collect image data of the target storage location.

[0010] In a possible implementation, the image acquisition device is provided on the transport device, and before controlling the image acquisition device to start and acquire image data of the target storage location, the method further includes:

[0011] Control the transport device to align with the target storage location.

[0012] In a possible implementation, if it is determined based on the image data of the target storage location that the execution condition of the transport task is met, controlling the transport device to execute the transport task includes:

[0013] The image data of the target storage location is detected and processed to determine the status information of the target storage location and / or the status information of the target object; based on the status information of the target storage location and / or the status information of the target object, if it is determined that the execution conditions of the transport task are met, the transport device is controlled to execute the transport task.

[0014] In one possible implementation, the status information of the target storage location includes at least one of the following:

[0015] Obstacle information on the transport path of the target storage location; size information of the target storage location; whether the target storage location is vacant.

[0016] In one possible implementation, the target object status information includes at least one of the following:

[0017] The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; and the deformation degree information of the target object.

[0018] In a possible implementation, the transport task is a pickup task, and the execution condition of the transport task includes at least one of the following:

[0019] There are no obstacles on the picking path of the target storage location; the identity information, posture information and size information of the target object meet the picking conditions; the degree of damage of the target object is within the first preset safety threshold range; the degree of deformation of the target object is within the second preset safety threshold range.

[0020] In a possible implementation, the transport task is a cargo release task, and the execution conditions of the transport task include at least one of the following:

[0021] The target storage location is free; the size of the target storage location meets the conditions for releasing goods; and there are no obstacles on the releasing path of the target storage location.

[0022] In a possible implementation, the method further includes:

[0023] According to the image data of the target storage location, if it is determined that the execution conditions of the transport task are not met, an error message is sent to the server, wherein the error message includes at least one of the following: status information of the target storage location, status information of the target object, and unmet execution condition items.

[0024] In a possible implementation, after sending the error information to the server, the method further includes:

[0025] According to the scheduling instructions of the server, the storage robot is controlled to perform corresponding error handling behavior.

[0026] In one possible implementation, the error handling behavior is any one of the following:

[0027] Stay at the current location and wait for instructions; move to the target point; skip the current transport task and execute the next transport task.

[0028] In a possible implementation, collecting image data of the target storage location by the image acquisition device includes at least one of the following:

[0029] The two-dimensional image data of the target storage location is collected by the first shooting device; the three-dimensional point cloud data of the target storage location is collected by the second shooting device; and the two-dimensional point cloud data of the target storage location is collected by the laser radar device.

[0030] In a possible implementation manner, before the image acquisition device acquires the image data of the target storage location corresponding to the handling task, the method further includes:

[0031] In response to the execution instruction of the transport task, the storage robot is controlled to move to the target storage location.

[0032] Another aspect of the present invention is to provide a control device for a warehouse robot, which is applied to the warehouse robot. The warehouse robot includes a handling device and an image acquisition device, including:

[0033] A data acquisition module, configured to acquire image data of a target storage location corresponding to a handling task through the image acquisition device;

[0034] The control module is configured to control the transport device to execute the transport task if it is determined that the execution conditions of the transport task are met based on the image data of the target storage location.

[0035] In a possible implementation, the data acquisition module is further configured to:

[0036] When the warehouse robot moves to the target position corresponding to the target storage location, the image acquisition device is controlled to start and collect image data of the target storage location; or, when the warehouse robot moves to a preset range around the target storage location, the image acquisition device is controlled to start and collect image data of the target storage location.

[0037] In a possible implementation, the image acquisition device is provided on the transport device, and the control module is further configured to:

[0038] Control the transport device to align with the target storage location.

[0039] In a possible implementation manner, the control module is further configured to:

[0040] The image data of the target storage location is detected and processed to determine the status information of the target storage location and / or the status information of the target object; based on the status information of the target storage location and / or the status information of the target object, if it is determined that the execution conditions of the transport task are met, the transport device is controlled to execute the transport task.

[0041] In one possible implementation, the status information of the target storage location includes at least one of the following:

[0042] Obstacle information on the transport path of the target storage location; size information of the target storage location; whether the target storage location is vacant.

[0043] In one possible implementation, the target object status information includes at least one of the following:

[0044] The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; and the deformation degree information of the target object.

[0045] In a possible implementation, the transport task is a pickup task, and the execution condition of the transport task includes at least one of the following:

[0046] There are no obstacles on the picking path of the target storage location; the identity information, posture information and size information of the target object meet the picking conditions; the degree of damage of the target object is within the first preset safety threshold range; the degree of deformation of the target object is within the second preset safety threshold range.

[0047] In a possible implementation, the transport task is a cargo release task, and the execution condition of the transport task includes at least one of the following:

[0048] The target storage location is free; the size of the target storage location meets the conditions for releasing goods; and there are no obstacles on the releasing path of the target storage location.

[0049] In a possible implementation manner, the control module is further configured to:

[0050] According to the image data of the target storage location, if it is determined that the execution conditions of the transport task are not met, an error message is sent to the server, wherein the error message includes at least one of the following: status information of the target storage location, status information of the target object, and unmet execution condition items.

[0051] In a possible implementation manner, the control module is further configured to:

[0052] According to the scheduling instructions of the server, the storage robot is controlled to perform corresponding error handling behavior.

[0053] In one possible implementation, the error handling behavior is any one of the following:

[0054] Stay at the current location and wait for instructions; move to the target point; skip the current transport task and execute the next transport task.

[0055] In one possible implementation, the data acquisition module is further configured to perform at least one of the following:

[0056] The two-dimensional image data of the target storage location is collected by the first shooting device; the three-dimensional point cloud data of the target storage location is collected by the second shooting device; and the two-dimensional point cloud data of the target storage location is collected by the laser radar device.

[0057] In a possible implementation, the control module is further configured to control the storage robot to move toward the target storage location in response to an execution instruction of the transport task.

[0058] Another aspect of the present invention is to provide a warehousing robot, comprising:

[0059] A transport device, an image acquisition device, a processor, a memory, and a computer program stored in the memory and executable on the processor;

[0060] Wherein, the processor implements the above-mentioned control method of the storage robot when running the computer program.

[0061] Another aspect of the present invention is to provide a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the control method of the storage robot described above is implemented.

[0062] The control method, device, equipment and readable storage medium of the warehouse robot provided by the present invention, before executing the handling task, collects image data of the target storage location corresponding to the handling task through an image acquisition device, and determines whether the execution conditions of the handling task are currently met based on the image data of the target storage location. When it is determined that the execution conditions of the handling task are met, that is, there is no danger when the handling device executes the handling task, the handling device is controlled to execute the handling task, which can avoid danger, improve the safety of cargo picking and placing, and reduce the probability of cargo damage and shelf tipping. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a flow chart of the control method of the storage robot provided in the first embodiment of the present invention;

[0064] Figure 2 Flowchart of the control method of the storage robot provided in the second embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of the structure of the control device of the storage robot provided in the third embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the structure of the storage robot provided in Example 5 of the present invention.

[0067] The above drawings illustrate specific embodiments of the present invention, which will be described in more detail below. These drawings and the accompanying description are not intended to limit the scope of the present invention in any way, but rather to illustrate the concept of the present invention to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0068] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0069] The terms "first," "second," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. In the description of the following embodiments, "plurality" means more than two, unless otherwise specifically defined.

[0070] The present invention is specifically applied to intelligent warehousing systems, which include a warehousing robot, a scheduling system, and a warehouse. The warehouse includes multiple storage locations for objects such as boxes and goods. The warehousing robot can replace workers in the transportation of goods. The scheduling system communicates with the warehousing robot. For example, the scheduling system can issue transportation tasks to the warehousing robot, and the warehousing robot can send task execution status information to the scheduling system, etc.

[0071] In existing intelligent warehousing systems, shelf vibrations and human error can cause bins to shift within storage or fall from shelves. Warehouse robots can collide with these bins when retrieving or passing through them, creating safety risks for warehouse robots when storing and retrieving bins.

[0072] The control method of the storage robot provided by the present invention is intended to solve the above technical problems.

[0073] The following describes in detail the technical solutions of the present invention and how the technical solutions of this application solve the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0074] Example 1

[0075] Figure 1 This is a flow chart of the control method for a warehouse robot provided in the first embodiment of the present invention. The method in this embodiment is applied to a warehouse robot. In other embodiments, the method can also be applied to other devices. This embodiment uses a warehouse robot as an example for schematic illustration. The execution subject of the method in this embodiment can be a processor for controlling a warehouse robot to perform a handling task, for example, it can be a processor of a terminal device mounted on the warehouse robot. Figure 1 As shown, the specific steps of this method are as follows:

[0076] Step S101: Capture image data of a target storage location corresponding to a handling task through an image acquisition device.

[0077] The transport task includes the information of the corresponding target storage location, the task type, and other information required to perform the current task. The types of transport tasks can include pick-up tasks and drop-off tasks.

[0078] The warehouse robot is equipped with a handling device for picking up and / or placing goods. The handling device refers to a device used to pick up goods from or place goods into a storage location, such as a fork.

[0079] An image acquisition device is a device installed on a storage robot that can capture image data of a target storage location. For example, the image acquisition device can be a 2D camera, a 3D camera, a lidar, or similar device. In this disclosure, a 2D camera refers to a camera that captures two-dimensional data. Common 2D cameras include standard color cameras and black-and-white cameras. A 3D camera refers to a camera that captures stereoscopic data. This camera utilizes structured light reflected from objects or the field of view difference between binocular cameras. Common 3D cameras include Kinect and RealSense.

[0080] Optionally, the image acquisition device can be set on the transport device of the warehouse robot. When the warehouse robot moves towards the target storage location, or when the warehouse robot moves near the target storage location, the image acquisition device installed on the transport device can collect image data of the target storage location.

[0081] Before executing the transport task, the processor may obtain image data of the target storage location to determine whether the execution conditions of the transport task are currently met based on the image data of the target storage location.

[0082] Specifically, the processor controls the image acquisition device to acquire image data of the target storage location and sends the image data of the target storage location to the processor. The processor receives the image data of the target storage location sent by the image acquisition device, thereby acquiring the image data of the target storage location in real time.

[0083] Step S102: If it is determined based on the image data of the target storage location that the execution conditions of the transport task are met, the transport device is controlled to execute the transport task.

[0084] After acquiring the image data of the target storage location, the processor can detect and process the image data of the target storage location to detect the status information of the target storage location and the status information of the objects in the target storage location; and determine whether the execution conditions of the handling task are currently met based on the status information of the target storage location and the status information of the objects in the target storage location.

[0085] For example, the status information of the target storage location may include whether the target storage location is available, its size, whether there are obstacles on the path of the handling device for picking up goods from or putting goods into the target storage location, etc. The status information of the objects in the target storage location may include their identity, size, posture, degree of damage, degree of deformation, etc.

[0086] In addition, the information detected based on the image data of the target storage location may vary according to the needs of the actual application scenario, and this embodiment does not specifically limit this.

[0087] If it is determined that the execution conditions of the transport task are met, it means that under the current conditions, there is no danger in the transport device performing the transport task, and the transport device is controlled to perform the transport task.

[0088] If it is determined that the execution conditions of the transport task are not met, it means that under the current conditions, the transport device may be dangerous when performing the transport task, and the transport device will not be controlled to perform the transport task to avoid the danger.

[0089] The embodiment of the present invention collects image data of the target storage location corresponding to the handling task through an image acquisition device before executing the handling task, and determines whether the execution conditions of the handling task are currently met based on the image data of the target storage location. When the execution conditions of the handling task are not met, the handling device may be in danger when executing the handling task, so the handling task is temporarily not executed to avoid danger, thereby improving the safety of the warehouse robot.

[0090] Example 2

[0091] Figure 2 This is a flow chart of the control method for the storage robot provided in the second embodiment of the present invention. Based on the above-mentioned embodiment one, in this embodiment, based on the image data of the target storage location, if it is determined that the execution conditions of the handling task are met, then the handling device is controlled to perform the handling task, including: detecting and processing the image data of the target storage location to determine the status information of the target storage location and / or the target object; based on the status information of the target storage location and / or the target object, if it is determined that the execution conditions of the handling task are met, then the handling device is controlled to perform the handling task. Furthermore, based on the image data of the target storage location, if it is determined that the execution conditions of the handling task are not met, then an error message is sent to the server. Figure 2 As shown, the specific steps of this method are as follows:

[0092] Step S201: In response to an execution instruction of a transport task, the storage robot is controlled to move to a target storage location corresponding to the transport task.

[0093] Among them, the execution instruction of the handling task can be the instruction information sent by the scheduling system to the warehouse robot to trigger the warehouse robot to perform the handling task.

[0094] A transport task includes the target location information, the task type, and other information required to execute the current task. The types of transport tasks can include pick-up tasks and drop-off tasks.

[0095] When receiving the execution instruction of the transport task, the processor controls the storage robot to move to the target storage location corresponding to the transport task according to the new location of the target storage location corresponding to the transport task.

[0096] In this embodiment, the target storage location refers to the storage location corresponding to the transport task, and the target object refers to the transport target of this transport task. For example, if the transport task is to pick up goods, the target storage location refers to the storage location from which the goods need to be picked up, and the goods and / or containers to be picked up are the target objects. If the transport task is to place goods, the object to be stored is the target object, and the target storage location refers to the storage location where the target object needs to be placed.

[0097] Step S202: Capture image data of the target storage location through an image acquisition device.

[0098] Among them, the warehouse robot is equipped with a handling device for picking up goods. The handling device refers to a device used to pick up goods from or put goods into the storage location, such as a fork.

[0099] An image acquisition device is a device installed on a warehouse robot that can capture image data of a target storage location. This device can be a black and white camera, a color camera, a depth camera, or other image sensors. For example, this device can be a 2D camera, a 3D camera, or a LiDAR.

[0100] Optionally, the image acquisition device can be set on the transport device of the warehouse robot. When the warehouse robot moves towards the target storage location, or when the warehouse robot moves near the target storage location, the image acquisition device installed on the transport device can collect image data of the target storage location.

[0101] Optionally, the image acquisition device can be set on the transport device of the warehouse robot, facing the front of the transport device, so that when the transport device is aligned with the target storage location, the image acquisition device can be aligned with the target storage location and can accurately capture the image data of the target storage location.

[0102] Furthermore, the processor may also control the transport device to align with the target storage location by analyzing the relative position between the current position of the transport device and the target storage location.

[0103] Exemplarily, when the storage robot moves to a target position corresponding to a target storage location, the processor controls the image acquisition device to start and acquire image data of the target storage location.

[0104] Furthermore, when the storage robot moves to the target position corresponding to the target storage location, the processor can control the transport device to move to the target storage location so that the image acquisition device installed on the transport device can be aligned with the target storage location.

[0105] For example, during the process of a warehouse robot moving to a target position corresponding to a target storage location, when the warehouse robot moves to a preset range around the target storage location, the processor controls the image acquisition device to start in advance and acquire image data of the target storage location, thereby obtaining the image data of the target storage location in advance and performing detection processing. This allows for early determination of whether the execution conditions of the handling task are currently met, thereby completing the handling task in advance and improving efficiency. The preset range can be set according to actual application scenarios and is not specifically limited in this embodiment.

[0106] In this embodiment, if it is determined based on the image data of the target storage location that the execution conditions of the transport task are met, the transport device is controlled to execute the transport task, which can be specifically implemented by the following steps S203-S204.

[0107] Step S203: Detect and process the image data of the target storage location to determine the status information of the target storage location and / or the status information of the target object.

[0108] The status information of the target location includes at least one of the following:

[0109] Obstacle information on the transport path to the target location; the dimensions of the target location; and whether the target location is available. The transport path includes a pickup path and / or a delivery path.

[0110] The target object's status information includes at least one of the following:

[0111] The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; and the deformation degree information of the target object.

[0112] In this embodiment, the information detected in this step may vary depending on the transport task. The information detected in this step only needs to be used in subsequent steps to determine whether the execution conditions of the current transport task are met. As long as the determination of whether the execution conditions of the current transport task are met is sufficient, the less information detected, the higher the efficiency.

[0113] Exemplarily, the detection processing of image data may include: image filtering, feature extraction, target segmentation, deep learning, point cloud filtering, point cloud extraction, point cloud clustering, point cloud segmentation, deep learning of point clouds and other algorithm processing, and may also include other image processing algorithms in the field of image processing, which will be described in detail in the subsequent step S204 and will not be repeated here.

[0114] Step S204: Determine whether the execution conditions of the transport task are met based on the status information of the target storage location and / or the status information of the target object.

[0115] Specifically, if the transport task is to pick up goods, the execution conditions of the transport task include at least one of the following:

[0116] There are no obstacles on the picking path of the target storage location; the identity information, posture information and size information of the target object meet the picking conditions; the degree of damage of the target object is within the first preset safety threshold range; the degree of deformation of the target object is within the second preset safety threshold range.

[0117] Among them, the picking path refers to the route that the warehouse robot takes when it moves to the shelf, the handling device takes the cargo box (or the target object in the cargo box) from the target storage location, and moves the cargo box (or target object) to the designated position on the warehouse robot (such as the cache position on the warehouse robot).

[0118] For example, taking the target object as a material box and the handling task as picking up the goods, if the posture and size of the material box meet the picking conditions and there are no obstacles in the picking path, the fork can be extended to pick up the goods.

[0119] If the handling task is to release the goods, the execution conditions of the handling task include at least one of the following:

[0120] The target storage location is free; the size of the target storage location meets the conditions for releasing goods; there are no obstacles on the release path of the target storage location.

[0121] For example, taking the target object as a material box and the handling task as releasing the goods, if the target storage location is empty, the size of the target storage location meets the material box size requirements, and there are no obstacles in the releasing path, the fork can be extended to release the goods.

[0122] Among them, the cargo delivery path refers to the route that the warehouse robot takes before it moves to the shelf, the handling device takes the cargo box (or the target object in the cargo box) from the designated position on the warehouse robot (such as the cache position on the warehouse robot), and moves the cargo box (or target object) to the target storage location (or the cargo box at the target storage location).

[0123] In a possible implementation, two-dimensional image data of the target storage location may be collected by a first photographing device.

[0124] The first shooting device may be a shooting device such as a 2D camera that can capture two-dimensional image data.

[0125] Specifically, by adjusting the position of the storage robot and / or the handling device where the first camera is located, and adjusting the installation position of the first camera on the storage robot, the target storage location is within the field of view of the first camera.

[0126] The processor controls the first camera to turn on so that the first camera captures images within its field of view. Since the target storage location is within the field of view of the first camera, the image data that can be captured by the first camera includes the target storage location, that is, the first camera can capture image data of the target storage location and send it to the processor.

[0127] For this embodiment, the step can be: the processor filters and reduces noise on the received image data, extracts areas in the image that meet specific conditions, and uses a deep learning algorithm to extract and separate targets in the image to identify the target storage location in the image, objects in the target storage location, and other obstacles and other targets, and determines whether the execution conditions of the handling task are currently met based on the image processing and recognition results; it can also be: using a deep learning method to perform image registration to determine whether the execution conditions of the handling task are met; this article does not impose any restrictions on this.

[0128] Among them, the filtering noise reduction processing can be the application of filtering algorithms such as Gaussian filtering, mean filtering, and median filtering.

[0129] For example, the specific conditions may include at least one of the following: a specific color, a position in an image, a pixel value, etc. The specific conditions may be set and adjusted according to the specific characteristics of the target to be identified in an actual application scenario, and are not specifically limited in this embodiment.

[0130] Exemplarily, feature extraction is performed on the image, and the extracted features may include at least one of the following: the edge lines of the container, feature points on the container surface, specific graphics on the container surface, and the color of the container surface. The feature extraction results may include at least one of the following: the enclosed area of ​​the container lines, the coordinates of the intersection points of the container lines, the number of feature points, the area of ​​the specific graphics, etc. Based on the feature extraction results, whether the container size meets the requirements can be determined by determining whether the size of the container lines meets a preset threshold. Alternatively, deep learning methods can be used to directly identify and determine whether the container size meets the requirements; by determining whether the specific graphics are preset graphics; or by determining whether the target is the target storage location specified in the handling task or the target object to be picked up.

[0131] In this embodiment, the features extracted include, the information included in the feature extraction results, and the rules for judging whether the execution conditions of the transport task are currently met based on the specific feature extraction results can be adjusted according to the actual application scenario, and this embodiment does not make specific limitations here.

[0132] In this implementation, any step can be increased or decreased, or the order can be changed based on the specific circumstances of the actual application scenario. Other algorithms can also be inserted based on the actual situation to improve the detection effect. This embodiment does not make specific limitations here.

[0133] In another possible implementation, three-dimensional point cloud data of the target storage location may be collected by a second photographing device.

[0134] The second shooting device may be a shooting device capable of collecting three-dimensional point cloud data, such as a 3D camera, a 3D laser radar, or a 2D laser radar. A 2D laser radar can obtain 3D point cloud data by moving.

[0135] Specifically, by adjusting the position of the storage robot and / or handling device where the second camera is located, and adjusting the installation position of the second camera on the storage robot, the target storage location is within the field of view of the second camera.

[0136] The processor controls the second camera to turn on so that the second camera captures images within its field of view. Since the target storage location is within the field of view of the second camera, the image data that can be captured by the second camera includes the target storage location, that is, the second camera can capture image data of the target storage location and send it to the processor.

[0137] For this implementation method, in this step, the processor processes the received three-dimensional point cloud data, performs noise reduction on the sampled point cloud, extracts the target area in the point cloud, clusters the point cloud, and uses the clustering results to determine whether there are obstacles in the current pick-up / drop-off path and whether the storage location size meets the execution conditions of the handling task; extracts the status information of the object (material box) in the target area, and uses the status information of the object (material box) to determine whether the object and the storage location meet the execution conditions of the handling task.

[0138] For example, the target region in the point cloud can be extracted based on whether the 3D coordinates of the point cloud fall within a preset spatial region. The preset spatial region can be set and adjusted according to actual application scenarios and is not specifically limited in this embodiment.

[0139] For example, if a point cloud object category exists in the target area after clustering, it is determined that there is an obstacle in the pickup / dropoff path or that the storage location dimensions do not meet the requirements for dropoff. Conversely, if a point cloud object category does not exist in the target area after clustering, it is determined that there are no obstacles in the pickup / dropoff path and that the storage location dimensions meet the requirements for dropoff.

[0140] Exemplarily, taking the object as a material box as an example, the state information of the object may include at least one of the following: posture, size, flatness, and texture.

[0141] For example, taking the object as a container, determining whether the object and the storage location meet the execution conditions of the transport task based on the state information of the object (container) includes at least one of the following:

[0142] Based on the status information of the material box, the material box within the field of view of the second camera device can be identified. If the status information of the material box within the field of view can be captured, it is considered that there is a material box in front; if the status of the material box in the field of view is empty, it is considered that there is no material box in front and the conditions for placing the goods are met; if the size of the material box is smaller than the size threshold, it is considered that the conditions for picking up the goods are met; if the current placement angle of the material box is within the safe range of the material box placement angle, it is considered that the conditions for picking up the goods are met.

[0143] Among them, the size threshold and the safe range of the material box placement angle can be set and adjusted according to the actual application scenario, and this embodiment does not make specific limitations here.

[0144] In this implementation, any step can be increased or decreased, or the order can be changed based on the specific circumstances of the actual application scenario. Other algorithms can also be inserted based on the actual situation to improve the detection effect. This embodiment does not make specific limitations here.

[0145] A third possible implementation method is to collect two-dimensional point cloud data of the target storage location through a laser radar device, or to obtain two-dimensional point cloud data through movement using a single-point laser rangefinder.

[0146] Specifically, by adjusting the position of the warehouse robot and / or handling device where the laser radar device is located, and adjusting the installation position of the laser radar device on the warehouse robot, the target storage location is within the field of view of the laser radar device.

[0147] The processor controls the laser radar device to turn on, so that the laser radar device scans the image within its field of view. Since the target storage location is within the field of view of the laser radar device, the image data that can be captured by the laser radar device includes the target storage location, that is, the laser radar device can capture the image data of the target storage location and send it to the processor.

[0148] For this implementation method, in this step, the processor processes the received two-dimensional point cloud data, performs noise reduction on the sampled point cloud, extracts the target area in the point cloud, clusters the point cloud, and uses the clustering results to determine whether there are obstacles in the current pick-up / drop-off path and whether the storage location size meets the execution conditions of the handling task; extracts the status information of the object (material box) in the target area, and uses the status information of the object (material box) to determine whether the object and the storage location meet the execution conditions of the handling task.

[0149] For example, if a point cloud object category exists in the target area after clustering, it is determined that an obstacle exists in the pickup / drop-off path, or that the storage location dimensions do not meet the requirements for drop-off. Conversely, if no point cloud object category exists in the target area after clustering, it is determined that no obstacle exists in the pickup / drop-off path. Furthermore, by calculating the length of the storage location's edges and the angle between them, and determining whether these meet preset length and angle thresholds, it is determined whether the storage location dimensions meet the requirements for drop-off. The length and angle thresholds can be determined based on the size of the container and are not specifically defined in this embodiment.

[0150] Exemplarily, taking the object as a material box as an example, the state information of the object may include at least one of the following: angle, size, and flatness.

[0151] For example, taking the object as a container, determining whether the object and the storage location meet the execution conditions of the transport task based on the state information of the object (container) includes at least one of the following:

[0152] Based on the container status information, the lidar device can identify containers within its field of view. If the container status information is captured, it is considered that there is a container ahead. If the container status is empty, it is considered that there is no container ahead, and the conditions for placing the container are met. If the container size is less than the size threshold, the conditions for picking up the container are met. If the current container placement angle is within the safe range for container placement angles, the conditions for picking up the container are met. The size threshold and the safe range for container placement angles can be set and adjusted according to the actual application scenario and are not specifically limited in this embodiment.

[0153] In this implementation, any step can be increased or decreased, or the order can be changed based on the specific circumstances of the actual application scenario. Other algorithms can also be inserted based on the actual situation to improve the detection effect. This embodiment does not make specific limitations here.

[0154] Step S205: If it is determined that the execution condition of the transport task is met, the transport device is controlled to execute the transport task.

[0155] In step S204, if the conditions for executing the transport task are determined to be met, then under the current conditions, the transport device can execute the transport task without causing any danger. The transport device is then controlled to execute the transport task. For example, the fork is controlled to extend to perform a pick-up action, removing a container from the target storage location; or the fork is controlled to extend to perform a drop-down action, placing a container at the target storage location.

[0156] Step S206: If it is determined that the execution conditions of the transport task are not met, an error message is sent to the server.

[0157] The error information includes at least one of the following: status information of the target storage location, status information of the target object, and unsatisfied execution condition items.

[0158] For example, there are obstacles in the picking / putting path of the warehouse, the posture of the material box exceeds the safe range, the size of the material box exceeds the set range, the degree of damage to the material box exceeds the threshold for safe picking, etc.

[0159] In the above step S204, if it is determined that the execution condition of the transport task is not met, it means that under the current conditions, the transport device may be dangerous when performing the transport task, and the transport device will not be controlled to perform the transport task to avoid danger.

[0160] Furthermore, the processor can send an error message to the dispatch system server so that the dispatch system can guide the human to complete the restoration of the warehouse robot's working condition. For example, the dispatch system can send a message to the terminal device of the corresponding technician to inform the worker how to complete the restoration.

[0161] For example, if the current handling task is a pickup task, the worker can be instructed to remove obstacles in the storage location, adjust the position of the container, remove severely damaged containers, etc. If the current handling task is a delivery task, the worker can be instructed to modify the size of the current storage location, remove obstacles in the storage location, remove containers in the storage location, etc.

[0162] Step S207: Control the storage robot to execute corresponding error handling behavior according to the scheduling instructions of the server.

[0163] In this embodiment, if it is determined that the execution conditions of the transport task are not met, the processor may further control the storage robot to execute corresponding error handling behavior according to the scheduling instructions of the server.

[0164] Where error handling behavior is any of the following:

[0165] Stay at the current location and wait for instructions; move to the target point; skip the current transport task and execute the next one.

[0166] Among them, staying at the current position and waiting for instructions means that the warehouse robot maintains the posture before performing the handling task (picking up or putting away goods), and does not perform any actions until the working conditions are restored, and stays on standby.

[0167] The target point is any point in the map that does not interfere with the movement of other robots. Optionally, the processor can control the warehouse robot to move to the target point closest to its current position to improve efficiency.

[0168] Skip the current transport task and execute the next one: This means giving up on getting the current container or giving up on storing the current container and entering the process of picking up / putting the next container.

[0169] In another implementation of this embodiment, if it is determined that the execution conditions of the transport task are not met, the processor can send an error message to the server and then control the device to execute a corresponding error handling behavior according to a preset error handling policy. In other words, an error handling policy configuration can be pre-set for the warehouse robot. When an error occurs during the execution of the transport task, the corresponding error handling behavior can be directly executed according to the preset error handling policy.

[0170] The embodiment of the present invention collects image data of the target storage location through the image acquisition device on the warehouse robot, and uses this as basic data for judging whether the execution conditions of the handling task are met. There is no need to set sensors on each storage location, and it can be flexibly applied to various types of storage systems, thereby improving the versatility and flexibility of the warehouse robot and greatly reducing the cost of construction and deployment. Furthermore, the warehouse robot can be directly applied to a variety of storage systems. Compared with the sonic radar, gravity meter and other sensors set in the existing storage locations, the image acquisition device in this embodiment (which can be a 2D camera, 3D camera, 3D laser radar, 2D laser radar, single-point laser rangefinder, etc.) collects 2D or 3D image data of the target storage location, and detects the target storage location and target material box based on these image data, thereby improving the detection accuracy, so that the situation where the execution conditions of the handling task are not met can be determined more accurately, and the occurrence of dangerous situations can be better avoided, thereby improving the safety of the warehouse robot.

[0171] Example 3

[0172] Figure 3 This is a schematic diagram of the structure of the control device of the storage robot provided in the third embodiment of the present invention. The control device of the storage robot provided in the embodiment of the present invention can execute the processing flow provided in the embodiment of the control method of the storage robot. Figure 3 As shown, the control device 30 of the storage robot includes: a control module 301 and a data acquisition module 302.

[0173] Specifically, the control module 301 is used to control the storage robot to move to the target storage location of the transport task in response to the execution instruction of the transport task;

[0174] The data acquisition module 302 is used to collect image data of the target storage location through an image acquisition device;

[0175] The control module 301 is further configured to: control the transport device to execute the transport task if it is determined that the execution conditions of the transport task are met based on the image data of the target storage location.

[0176] The device provided in the embodiment of the present invention can be specifically used to execute the method embodiment provided in the above-mentioned embodiment 1, and the specific functions will not be repeated here.

[0177] The embodiment of the present invention collects image data of the target storage location through an image acquisition device before executing the handling task, and determines whether the execution conditions of the handling task are currently met based on the image data of the target storage location. When it is determined that the execution conditions of the handling task are not met, the handling device may be in danger when executing the handling task, so the handling task is temporarily not executed to avoid danger, thereby improving the safety of the warehouse robot.

[0178] Example 4

[0179] On the basis of the above-mentioned embodiment 3, in this embodiment, the control module is further configured to:

[0180] The image data of the target storage location is detected and processed to determine the status information of the target storage location and / or the target object; based on the status information of the target storage location and / or the target object, if it is determined that the execution conditions of the handling task are met, the handling device is controlled to execute the handling task.

[0181] In a possible implementation, the data acquisition module is further configured to:

[0182] When the warehouse robot moves to the target position corresponding to the target storage location, the image acquisition device is controlled to start and collect image data of the target storage location; or, during the process of the warehouse robot moving to the target position corresponding to the target storage location, the image acquisition device is controlled to start and collect image data of the target storage location.

[0183] In a possible implementation, the image acquisition device is provided on the transport device, and the control module is further configured to:

[0184] Control the handling device to align with the target storage location.

[0185] In one possible implementation, the status information of the target storage location includes at least one of the following:

[0186] Obstacle information on the pick-up / drop-off path of the target storage location; size information of the target storage location; whether there are objects placed in the target storage location.

[0187] In one possible implementation, the target object's state information includes at least one of the following:

[0188] The identity information of the target object; the posture information of the target object; the size information of the target object; the damage degree information of the target object; and the deformation degree information of the target object.

[0189] In one possible implementation, if the transport task is to pick up goods, the execution condition of the transport task includes at least one of the following:

[0190] There are no obstacles on the picking path of the target storage location; the identity, posture and size of the target object meet the picking conditions; the degree of damage of the target object is within the first preset safety threshold; the degree of deformation of the target object is within the second preset safety threshold.

[0191] In one possible implementation, if the transport task is to release goods, the execution condition of the transport task includes at least one of the following:

[0192] The target storage location is free; the size of the target storage location meets the conditions for releasing goods; there are no obstacles on the release path of the target storage location.

[0193] In a possible implementation manner, the control module is further configured to:

[0194] If it is determined based on the image data of the target storage location that the execution conditions of the transport task are not met, an error message is sent to the server, where the error message includes at least one of the following: status information of the target storage location, status information of the target object, and unmet execution conditions.

[0195] In a possible implementation manner, the control module is further configured to:

[0196] According to the server's scheduling instructions, the warehouse robot is controlled to perform corresponding error handling behaviors.

[0197] In one possible implementation, the error handling behavior is any of the following:

[0198] Stay at the current location and wait for instructions; move to the target point; skip the current transport task and execute the next one.

[0199] In one possible implementation, the data acquisition module is further configured to perform at least one of the following:

[0200] The first shooting device collects two-dimensional image data of the target storage location; the second shooting device collects three-dimensional point cloud data of the target storage location; and the laser radar device collects two-dimensional point cloud data of the target storage location.

[0201] The device provided in this embodiment of the present invention can be specifically used to execute the method embodiment provided in the above-mentioned embodiment 2, and the specific functions will not be described in detail here.

[0202] The embodiment of the present invention collects image data of the target storage location through the image acquisition device on the warehouse robot, and uses this as basic data for judging whether the execution conditions of the handling task are met. There is no need to set sensors on each storage location, and it can be flexibly applied to various types of storage systems, thereby improving the versatility and flexibility of the warehouse robot and greatly reducing the cost of construction and deployment. Furthermore, the warehouse robot can be directly applied to a variety of storage systems. Compared with the sonic radar, gravity meter and other sensors set in the existing storage locations, the image acquisition device in this embodiment (which can be a 2D camera, 3D camera, 3D laser radar, 2D laser radar, single-point laser rangefinder, etc.) collects 2D or 3D image data of the target storage location, and detects the target storage location and target material box based on these image data, thereby improving the detection accuracy, so that the situation where the execution conditions of the handling task are not met can be determined more accurately, and the occurrence of dangerous situations can be better avoided, thereby improving the safety of the warehouse robot.

[0203] Example 5

[0204] Figure 4 This is a schematic diagram of the structure of the storage robot provided by the fifth embodiment of the present invention. Figure 4 As shown, the device 100 includes: a processor 1001 , a memory 1002 , and a computer program stored in the memory 1002 and executable on the processor 1001 .

[0205] Among them, when the processor 1001 runs the computer program, it implements the control method of the storage robot provided by any of the above method embodiments.

[0206] The embodiment of the present invention collects image data of the target storage location through the image acquisition device on the warehouse robot, and uses this as basic data for judging whether the execution conditions of the handling task are met. There is no need to set sensors on each storage location, and it can be flexibly applied to various types of storage systems, thereby improving the versatility and flexibility of the warehouse robot and greatly reducing the cost of construction and deployment. Furthermore, the warehouse robot can be directly applied to a variety of storage systems. Compared with the sonic radar, gravity meter and other sensors set in the existing storage locations, the image acquisition device in this embodiment (which can be a 2D camera, 3D camera, 3D laser radar, 2D laser radar, single-point laser rangefinder, etc.) collects 2D or 3D image data of the target storage location, and detects the target storage location and target material box based on these image data, thereby improving the detection accuracy, so that the situation where the execution conditions of the handling task are not met can be determined more accurately, and the occurrence of dangerous situations can be better avoided, thereby improving the safety of the warehouse robot.

[0207] In addition, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the control method of the storage robot provided by any of the above method embodiments is implemented.

[0208] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0209] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof, which is limited only by the appended claims.

Claims

1. A control method for a warehouse robot, wherein the warehouse robot has a handling device and an image acquisition device, characterized in that: include: Collecting image data of the target storage location corresponding to the handling task by the image acquisition device; Performing detection and processing on the image data of the target storage location to determine status information of the target storage location, wherein the status information of the target storage location includes obstacle information on a transport path of the target storage location or size information of the target storage location; If it is determined that the execution condition of the transport task is met according to the determined state information, controlling the transport device to execute the transport task; When the transport task is a cargo release task, the execution condition of the transport task includes at least one of the following: The target storage location is idle; The size of the target storage location meets the conditions for releasing goods; There are no obstacles on the delivery path to the target storage location; Among them, the cargo release path is the route that the warehouse robot takes before moving to the shelf, the handling device takes the cargo box from the cache position on the warehouse robot, and moves the cargo box to the target storage location, or the route that the handling device takes the target object in the cargo box from the cache position on the warehouse robot, and moves the target object to the cargo box at the target storage location.

2. The method according to claim 1, characterized in that The collecting of image data of the target storage location corresponding to the handling task by the image collecting device includes: When the storage robot moves to the target position corresponding to the target storage location, controlling the image acquisition device to start and acquire image data of the target storage location; or, When the storage robot moves to a preset range around the target storage location, the image acquisition device is controlled to start and acquire image data of the target storage location.

3. The method according to claim 2, characterized in that The image acquisition device is provided on the transport device, and before controlling the image acquisition device to start and acquire image data of the target storage location, the method further includes: The transport device is controlled to align with the target storage location.

4. The method according to claim 1, wherein When the transport task is a pickup task, After detecting and processing the image data of the target storage location, determining at least one of status information of the target storage location and status information of the target object, wherein the status information of the target storage location includes obstacle information on a transportation path of the target storage location or size information of the target storage location, and the status information of the target object includes damage degree information of the target object or deformation degree information of the target object; The execution conditions of the pickup task include at least one of the following: There are no obstacles on the path to the target location; The posture information or size information of the target object meets the pickup conditions; The damage degree of the target object is within a first preset safety threshold range; as well as The deformation degree of the target object is within a second preset safety threshold range.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: If it is determined based on the image data of the target storage location that the execution condition of the transport task is not met, an error message is sent to the server, wherein the error message includes at least one of the following: status information of the target storage location and an unmet execution condition item.

6. The method according to claim 5, characterized in that After sending the error message to the server, it also includes: According to the scheduling instructions of the server, the storage robot is controlled to perform corresponding error handling behavior.

7. The method according to claim 6, characterized in that The error handling behavior is any of the following: Stay at the current location and wait for instructions; Move to the target point; Skip the current transport task and execute the next transport task.

8. The method according to any one of claims 1 to 4, characterized in that Collecting image data of the target storage location by the image acquisition device includes at least one of the following: Collecting two-dimensional image data of the target storage location by a first shooting device; Collecting three-dimensional point cloud data of the target storage location by a second shooting device; The two-dimensional point cloud data of the target storage location is collected by a laser radar device.

9. The method according to claim 8, characterized in that The first shooting device is a shooting device for collecting two-dimensional image data, wherein the target storage location is within the field of view of the first shooting device by adjusting the position of at least one of the warehouse robot and the handling device where the first shooting device is located, and adjusting the installation position of the first shooting device on the warehouse robot.

10. The method according to claim 8, characterized in that The second shooting device is a shooting device for collecting three-dimensional point cloud data, wherein the target storage location is within the field of view of the second shooting device by adjusting the position of at least one of the warehouse robot and the transport device where the second shooting device is located, and adjusting the installation position of the second shooting device on the warehouse robot.

11. The method according to claim 8, characterized in that By adjusting the position of at least one of the warehouse robot and the transport device where the laser radar device is located, and adjusting the installation position of the laser radar device on the warehouse robot, the target storage location is within the field of view of the laser radar device.

12. The method according to any one of claims 1 to 4, characterized in that Before collecting the image data of the target storage location corresponding to the handling task by the image acquisition device, the method further includes: In response to the execution instruction of the transport task, the storage robot is controlled to move to the target storage location.

13. A control device for a storage robot, characterized in that: Applied to a warehouse robot, the warehouse robot includes a handling device and an image acquisition device, including: A data acquisition module, configured to acquire image data of a target storage location corresponding to a handling task through the image acquisition device; a control module, configured to detect and process the image data of the target storage location to determine status information of the target storage location, wherein the status information of the target storage location includes obstacle information on a transport path of the target storage location or size information of the target storage location; If it is determined that the execution condition of the transport task is met according to the determined state information, controlling the transport device to execute the transport task; When the transport task is a cargo release task, the execution condition of the transport task includes at least one of the following: The target storage location is idle; The size of the target storage location meets the conditions for releasing goods; There are no obstacles on the delivery path to the target storage location; Among them, the cargo release path is the route that the warehouse robot takes before moving to the shelf, the handling device takes the cargo box from the cache position on the warehouse robot, and moves the cargo box to the target storage location, or the route that the handling device takes the target object in the cargo box from the cache position on the warehouse robot, and moves the target object to the cargo box at the target storage location.

14. A storage robot, characterized in that: include: a processor, a memory, and a computer program stored on the memory and executable on the processor; Wherein, when the processor runs the computer program, the method according to any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images