Intelligent warehouse-out system, method and equipment for robot three-dimensional warehouse and medium

Abstract

The invention relates to an intelligent warehouse-out system, method, equipment and medium for a robot stereoscopic warehouse, and the system is used for replacing a tool corresponding to a task instruction according to the task instruction through an execution device, and executing an operation corresponding to the task instruction through the tool; the recognition device is used for shooting the box body through a 3D camera to obtain a 3D image, shooting through a 2D camera to obtain color information, and converting a depth map in the 3D image into a three-dimensional coordinate in a world coordinate system; determining the position, size and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image; the transportation equipment is used for transporting the materials to a specified position according to the received transportation instruction; and the scheduling processing module is used for receiving the upper-layer control instruction and distributing the control instruction to the corresponding execution device, the recognition device and the transportation equipment. The problem that an intelligent warehouse-out system of a robot three-dimensional warehouse is low in efficiency in the prior art is solved.

Description

Technical Field

[0001] This application relates to the technical field of intelligent outbound of robot stereoscopic warehouses, and particularly to an intelligent outbound system, method, device and medium for robot stereoscopic warehouses. Background Art

[0002] With the development of science and technology and the progress of society, computer technology has developed rapidly, and new requirements have been put forward for warehouse management methods. Especially in the field of military material management, the traditional manual maintenance method has been difficult to meet the needs of high efficiency, accuracy and rapid response. Manual operations have problems such as difficult material search and inconvenient access to goods, which not only affect work efficiency but also increase the possibility of errors.

[0003] Currently, the existing intelligent warehouse outbound systems on the market often have single interaction and limited functions, and are inefficient to use. Therefore, the intelligent outbound systems in the existing related technologies have the problem of low efficiency. Summary of the Invention

[0004] Embodiments of this application provide an intelligent outbound system, method, device and medium for a robot stereoscopic warehouse, so as to at least solve the problem of low efficiency in the intelligent outbound system of the robot stereoscopic warehouse in related technologies.

[0005] In a first aspect, embodiments of this application provide an intelligent outbound system for a robot stereoscopic warehouse. The system includes an execution device, an identification device, a transportation device and a scheduling and processing module; wherein,

[0006] The execution device is configured to replace a tool corresponding to the task instruction according to the task instruction, and execute an operation corresponding to the task instruction through the tool;

[0007] The identification device is configured to capture a box by a 3D camera to obtain a 3D image, and capture color information by a 2D camera, and convert a depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determine the position, size and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image;

[0008] The transportation device is configured to transport the material to a specified position according to the received transportation instruction;

[0009] The scheduling and processing module is configured to receive an upper-layer control instruction, and distribute the control instruction to the corresponding execution device, identification device and transportation device.

[0010] In an embodiment, when the identification device obtains the size and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image, it is configured to:

[0011] Define a region of interest (ROI) including the material to be recognized in the 3D image, and extract the point cloud data within the ROI;

[0012] Generate a planar mask for labeling the point cloud data within the ROI based on the point cloud data;

[0013] Use the planar mask to convert the extracted point cloud data into three-dimensional coordinates in the world coordinate system;

[0014] In the three-dimensional coordinates, identify the shape of the target material that meets the predetermined parameters through a shape detection algorithm;

[0015] Convert the shape of the target material to the 2D image through the internal and external parameters of the 3D camera, and combine the color information of the 2D image. Use image processing techniques to perform color classification on the shape of the target material to determine the color of the shape of the target material;

[0016] Based on the shape of the target material and the color of the shape of the target material, determine the position, size, and color information of the shape of the target material.

[0017] In one embodiment, when converting the depth map in the 3D image to three-dimensional coordinates in the world coordinate system, the recognition device is used for:

[0018] Obtain the internal and external parameters of the 3D camera by calibrating the 3D camera;

[0019] Use the 3D camera to obtain the depth map of the real scene, and convert the pixel coordinates of the depth map into three-dimensional coordinates in the world coordinate system through the internal and external parameters;

[0020] Calculate the actual distance of the center point in the world coordinate system and the depth value of the center point of the depth map, calculate the ratio of the actual distance to the depth value, and obtain the conversion ratio between the depth image pixels and the real actual measurement based on the ratio;

[0021] Based on the conversion ratio, obtain the actual length of the material in the real world through the pixel distance of the depth map.

[0022] In one embodiment, the execution device includes at least one manipulator, and the manipulator replaces the tool corresponding to the task instruction according to the task instruction and performs operations of grasping, transporting, and placing materials; wherein,

[0023] The tool includes a fixture and a suction device for grasping different types of materials.

[0024] In one embodiment, the transportation device includes an AGV transport vehicle, and the AGV transport vehicle is used for automatically transporting materials in the warehouse according to the task instruction; wherein,

[0025] The AGV transport vehicle includes a control panel, a guiding sensor, a direction potentiometer, a status indicator light, an obstacle avoidance sensor, a photoelectric control signal sensor, a driving unit, a guiding magnetic strip, and a power supply.

[0026] In one embodiment, the system further includes: the scheduling and processing module further includes a task management unit, and the task management unit supervises the task execution situation, records the task completion status, and issues an alarm when an exception occurs.

[0027] In one embodiment, the system includes an electric control cabinet, and the electric control cabinet includes a power cable, a programmable logic controller (PLC), a servo driver, and an encoder. The electric control cabinet is used to monitor and control the operating status of the system.

[0028] In a second aspect, an embodiment of the present application provides a method for intelligent outbound of a robot three-dimensional warehouse. The method includes:

[0029] Receiving an outbound task instruction from the upper layer control, where the task instruction includes the required material information;

[0030] Taking a 3D image of the material through the 3D camera, and obtaining color information by taking a 2D image of the material; converting the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determining the position, size, and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image;

[0031] Receiving and parsing the task instruction through the scheduling and processing module, and allocating corresponding tasks to the execution device, the recognition device, and the transportation device according to the task instruction;

[0032] According to the task instruction, the execution device switches to the tool corresponding to the task instruction, and performs operations of grasping, transporting, and placing the material through the tool;

[0033] The transportation device transports the material from the storage location to the designated location according to the received transportation instruction;

[0034] After the material is successfully transported to the designated location, confirm that the task is completed, and record and update the system status.

[0035] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements a robot three-dimensional warehouse intelligent outbound system as described in the first aspect above.

[0036] Fourthly, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements a robot three-dimensional library intelligent outbound system as described in the first aspect above.

[0037] The robot three-dimensional library intelligent outbound system, method, device and medium provided by the embodiments of the present application at least have the following technical effects.

[0038] Through an execution device, which is used to replace a tool corresponding to a task instruction according to the task instruction and perform an operation corresponding to the task instruction through the tool; an identification device, which is used to capture a box by a 3D camera to obtain a 3D image, and capture color information by a 2D camera, and convert the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determine the position, size and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image; a transportation device, which is used to transport the material to a specified position according to the received transportation instruction; a scheduling processing module, which is used to receive an upper-layer control instruction and allocate the control instruction to the corresponding execution device, identification device and transportation device. It greatly improves the efficiency of the military product warehouse during inbound and outbound, realizes the automation of inbound and outbound of the vault, reduces the labor cost, and increases the accuracy. It solves the problem of low efficiency in the robot three-dimensional library intelligent outbound system in the related technology.

[0039] The details of one or more embodiments of the present application are set forth in the following drawings and description, so that the other features, objects and advantages of the present application will become more clearly understood. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0041] Figure 1 is a system structure block diagram of a robot three-dimensional library intelligent outbound shown according to an exemplary embodiment;

[0042] Figure 2 is a flowchart of a robot three-dimensional library intelligent outbound shown according to an exemplary embodiment;

[0043] Figure 3 is a structure block diagram of an electronic device shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be described and explained below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments provided in the present application without making creative efforts fall within the scope of protection of the present application.

[0045] Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar scenarios based on these drawings without making creative efforts. In addition, it can also be understood that although the efforts made in this development process may be complex and time-consuming, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes made on the basis of the technical content disclosed in the present application are only conventional technical means and should not be understood as the content disclosed in the present application being insufficient.

[0046] Referring to "embodiments" in the present application means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those of ordinary skill in the art explicitly and implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the ordinary meanings understood by persons with ordinary skills in the technical field to which this application belongs. The words such as "a", "an", "one kind", "the" and the like involved in this application do not indicate a quantity limitation and may represent a singular or plural number. The terms "include", "comprise", "have" and any variations thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may further include steps or units not listed, or may further include other steps or units inherent to these processes, methods, products or devices. The similar words such as "connect", "be connected", "couple" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the front and rear associated objects. The terms "first", "second", "third" and the like involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0048] The following are the noun interpretations for this application:

[0049] WCS (Warehouse Control System) workstation. WCS is a real-time control system used to coordinate the operations of all automated equipment in the warehouse.

[0050] AGV (Automated Guided Vehicle). AGV is an unmanned automatic transport vehicle that can move autonomously on a preset path. It is usually used for tasks such as material handling and cargo transportation.

[0051] RCS (Robot Control System). RCS is the brain of the whole system, responsible for managing and scheduling automated equipment such as AGVs and robots. It schedules AGVs to perform handling tasks based on business logic and closely cooperates with WCS to ensure the efficient operation of the logistics process.

[0052] There are the following several main problems in the current military product warehouse outbound system:

[0053] When manually conducting the inbound and outbound inspection ledger and distribution work, it is necessary to ensure that the accounts are in line, which requires relatively high requirements for the staff. Once an error occurs, it will have a serious impact.

[0054] Managers need to keep track of the storage status and location of materials in the warehouse at all times to ensure timely supply to higher-level requirements and give full play to the material turnover efficiency.

[0055] Before the materials are put into the warehouse, it is usually necessary to manually carry the materials to the designated location and place them on the shelves, which is a cumbersome and time-consuming operation process.

[0056] During the outbound process, it also relies on manual handling of materials onto pallets and then transported away by forklifts, which increases additional labor costs and also requires good teamwork.

[0057] During the handling process, it is also necessary to manually confirm the parameter data for matching, which is very time-consuming and error-prone.

[0058] In summary, the existing intelligent warehouse outbound systems on the market often have a single interaction, limited functions, and low efficiency in use. Therefore, there is a problem of low efficiency in the existing intelligent outbound systems in related technologies.

[0059] In view of the above problems, in order to improve the outbound efficiency of military product warehouses and enhance the speed and accuracy of information processing, a robot stereoscopic warehouse intelligent outbound system, method, device, and medium have been developed.

[0060] In a first aspect, an embodiment of the present application provides a system for intelligent outbound of a robot stereoscopic warehouse. Figure 1 It is a block diagram of a system for intelligent outbound of a robot stereoscopic warehouse shown according to an exemplary embodiment. As Figure 1 shown, the system includes an execution device 110, an identification device 120, a transportation device 130, and a scheduling and processing module 140; wherein,

[0061] The execution device 110 is configured to replace the tool corresponding to the task instruction according to the task instruction, and perform an operation corresponding to the task instruction through the tool.

[0062] The identification device 120 is configured to capture the box body through a 3D camera to obtain a 3D image, and capture color information through a 2D camera, and convert the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determine the position, size, and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image.

[0063] The transportation device 130 is configured to transport the material to the designated location according to the received transportation instruction.

[0064] The scheduling and processing module 140 is configured to receive the upper-layer control instruction and distribute the control instruction to the corresponding execution device, identification device, and transportation device.

[0065] In summary, an intelligent outbound system for a robotic stereoscopic warehouse provided by an embodiment of the present application integrates a transportation device, an execution device, an identification device, and a scheduling and processing module, aiming to achieve a high degree of automation in warehouse management, simplify the operation process, reduce the error rate, and support customized services, thereby solving the problem of low efficiency in the intelligent outbound system of a robotic stereoscopic warehouse in related technologies.

[0066] In one embodiment, the execution device 110 is configured to replace the tool corresponding to the task instruction according to the task instruction, and perform an operation corresponding to the task instruction through the tool.

[0067] Optionally, the execution device includes at least one manipulator, which replaces the tool corresponding to the task instruction according to the task instruction and performs operations of grasping, transporting, and placing materials; wherein,

[0068] The tool includes a fixture and a suction cup, which are used to grasp different types of materials.

[0069] Specifically, the robotic arm adopts a multi-axis linkage design to achieve flexible spatial positioning capabilities. The tool changing mechanism realizes the automatic replacement of tools through a quick connector to ensure operation efficiency. The fixture is controlled to open and close by a clamping plate servo drive motor, and an obstacle avoidance insulating block is installed on the clamping plate to prevent collision or short circuit with other objects during the grasping process. The suction cup uses the principle of vacuum adsorption and is suitable for grasping materials with a flat surface, such as cardboard boxes. When the WCS sends a task instruction to the execution device 110, it first determines the type of material to be processed. According to the material type, the execution device 110 synchronizes the tool information currently carried and decides whether to replace it with the corresponding tool. After completing the tool replacement, the manipulator will move to the specified position and use the tool on the TCP (Tool Center Point) to grasp the target material. After the grasping is completed, according to the storage location number information provided by the WCS, the manipulator will transport the material to the specified target position and place it. Throughout the process, the execution device 110 continuously communicates with the WCS to ensure that each operation meets the task requirements.

[0070] In one embodiment, when converting the depth map in the 3D image into three-dimensional coordinates in the world coordinate system, the identification device is used for:

[0071] Obtaining the internal and external parameters of the 3D camera through calibration of the 3D camera;

[0072] Using the 3D camera to obtain the depth map of the real scene, and converting the pixel coordinates of the depth map into three-dimensional coordinates in the world coordinate system through the internal and external parameters;

[0073] Calculate the actual distance of the center point of the world coordinate system and the depth value of the center point of the depth map, calculate the ratio of the actual distance to the depth value, and obtain the conversion ratio between the depth image pixels and the real-world actual measurement based on the ratio;

[0074] Based on the conversion ratio, obtain the actual length of the material in the real world through the pixel distance of the depth map.

[0075] Optionally, to obtain the real-world length of the material from the depth map, to obtain the length of the object in the world coordinate system from the depth map, first calibrate the depth camera to obtain the internal and external camera parameters, and then use the depth camera to capture the depth map of the scene. Convert the pixel coordinates of the depth map into three-dimensional coordinates in the world coordinate system through the internal and external camera parameters. Calculate the distance of the center point of the world coordinate system (the world coordinate system refers to the coordinate system based on the base coordinates of the robot's first axis) and the depth value at the corresponding pixel position of the depth map, and obtain the conversion ratio (PPM, Pixels PerMetric) between the pixels and the real-world actual measurement through the ratio of the two (first obtain the relevant parameters of the camera, such as focal length, sensor size, and resolution, calculate the center point of the depth map through the data of the depth map, then the depth value of the center point can be obtained, and the PPM can be obtained through the internal parameters of the camera and geometric relationships). Finally, the actual length of the object in the real world can be obtained through the pixel distance of the depth map. It should be noted that the actual length of the material in the real world will be compared with the prior size of the issued task. After obtaining the size, the result will be output according to the coordinates of the object based on the camera coordinates. Through the analysis of the result given by vision, the pose will be sent to the robotic arm for absolute movement through algorithms and other customized functions. Specifically, the result given by the vision algorithm will be stored in the backend of the robot control software, and then according to the customized functions configured in the front end, including but not limited to eccentric grasping, grasping the edge after moving down a certain height, etc. It is equivalent to that after the software receives the result of the vision algorithm, it will adjust the result according to the front-end requirements and then send it to the robotic arm for movement.

[0076] By calibrating the 3D camera to obtain accurate internal and external parameters, it ensures the accurate conversion from the depth map to the world coordinate system, thereby improving the accuracy of material length measurement. It is applicable to various types of materials as long as the depth map can be obtained through the 3D camera, and has high versatility and flexibility.

[0077] In one embodiment, when the recognition device obtains the size and color information of the material by combining the three-dimensional coordinates of the 3D image and the color information of the 2D image, it is used for:

[0078] Define an area of interest in the 3D image that includes the material to be recognized, and extract the point cloud data within the area of interest;

[0079] Generate a plane mask for marking the point cloud data within the area of interest according to the point cloud data;

[0080] Using a planar mask, convert the extracted point cloud data into three-dimensional coordinates in the world coordinate system;

[0081] In the three-dimensional coordinates, identify the shape of the target material that meets the predetermined parameters through a shape detection algorithm;

[0082] Convert the shape of the target material to a 2D image through the internal and external parameters of the 3D camera, and combine the color information of the 2D image. Use image processing technology to classify the color of the shape of the target material to determine the color of the shape of the target material;

[0083] Based on the shape of the target material and the color of the shape of the target material, determine the position, size, and color information of the shape of the target material.

[0084] Optionally, first use a 3D camera and a 2D camera to photograph the box to obtain a 3D image and a 2D image. According to the input 3D and 2D images, the algorithm operation process is as follows:

[0085] First, according to the ROI (region of interest) information and the 3D image, obtain the surface point cloud and mask of the box, and extract a plane based on this region. Among them, according to the mask region corresponding to the ROI region, extract the corresponding regional plane point cloud data.

[0086] Second, find the current highest layer based on the extracted plane and retain the regions within a certain range above and below. Specifically, the current highest layer refers to the highest region after the point cloud in the depth map mask region is converted to the world coordinate system. The regions within a certain range above and below refer to the regions obtained by taking a certain distance above and below the highest region.

[0087] Third, identify the required rectangular blocks from this planar region through algorithms such as rectangular detection, and initially determine them as the surface of the box. Specifically, identify the required rectangular blocks, and the specific parameters are set internally by the algorithm. For the algorithm, there is no classification of rectangles, squares, rhombuses, etc., and only the shapes that meet the set parameters will be identified. The rectangular block is just the shape of the box that needs to be identified in this project. Rectangles with dimensions, aspect ratios, edge angles, etc. that meet the threshold requirements.

[0088] Fourth, transfer the rectangular blocks on these recognized 3D images to the 2D image through the internal and external parameters of the 3D and 2D cameras, so as to obtain the mask of the 2D image. Input the obtained 2D image mask area and the 2D image into the deep learning model for color classification detection to obtain the box color and the rectangular frame. Specifically, the deep learning model needs to use the 2D image to recognize the box color, and in order to exclude the interference of the irrelevant background on the 2D image, it is also necessary to select the ROI area on the 2D image. At this time, we already have the rectangular block area of the 3D image. However, the resolutions of the 2D image and the 3D image are different, so the rectangular area of the 3D image cannot be directly used on the 2D image. Therefore, the rectangular area of the 3D image needs to be transferred to the world coordinate system through the internal and external parameters of the 3D camera. Since the 2D and 3D cameras will be calibrated before shooting to obtain the relative position relationship between the two in the world coordinate system, the rectangular frame of the 3D image transferred to the world coordinate system can then be transferred to the 2D image through the internal and external parameters of the 2D camera, so that it can coincide with the box area on the 2D image. The 3D plus 2D image acquisition method can identify the shape more accurately and quickly through the 3D image. However, the 3D image does not contain color information. Therefore, it is necessary to add the 2D image for color recognition. And because the 2D image is added, during the process of the deep learning color recognition, the information of the recognized rectangular frame will also be output, which is equivalent to adding an extra layer of insurance to the results of the 3D image and can be used for comparison and verification.

[0089] Fifth, integrate the 3D image rectangular detection and 2D image color detection results, and output the recognized box body. Specifically, the box body surface detection result gives the length, width, height of the box body and the world coordinates of the four corners of the box body. The box body color detection result returns the color and coordinates of each box, and finally only classifies them into gray and non-gray. Correspond the box body coordinate position of the surface detection result with the coordinates of the color detection result, that is, find the position and color of each box, and output a signal indicating whether it is gray for each box.

[0090] The camera takes pictures to recognize the point clouds of the external dimensions of different materials, combines the model point cloud data collected by the algorithm for relative matching, and sends the position to the robot. The robot unpacks the box and places it on the shelf. The unique 3D plus 2D point cloud image acquisition and recognition method is different from the ordinary recognition on the market, with higher efficiency and accuracy.

[0091] In one embodiment, the transportation device 130 is configured to transport the material to a specified position according to the received transportation instruction.

[0092] Optionally, the transportation device includes an AGV transport vehicle, and the AGV transport vehicle is used to automatically transport materials in the warehouse according to the task instruction; wherein, the AGV transport vehicle includes a control panel, a guiding sensor, a direction potentiometer, a status indicator light, an obstacle avoidance sensor, a photoelectric control signal sensor, a driving unit, a guiding magnetic strip and a power supply.

[0093] Specifically, the AGV transport vehicle is an unmanned transport vehicle, which is a transport vehicle with safety protection and various transplant functions. The AGV travel control system includes a control panel for receiving and processing transport instructions from the warehouse control system and controlling the operation of the AGV transport vehicle, a guiding sensor for detecting guiding magnetic strips or landmarks to ensure that the AGV transport vehicle travels along a predetermined path, a direction potentiometer for adjusting the direction of the AGV transport vehicle to ensure that it travels along the correct route, a status indicator light for displaying the working status of the AGV transport vehicle, such as normal operation, failure, charging, etc., an obstacle avoidance sensor for detecting obstacles ahead and automatically stopping or bypassing when encountering obstacles, a photoelectric control signal sensor for receiving and sending photoelectric signals to achieve communication with other devices, a drive unit including a motor and wheels for driving the AGV transport vehicle to move, a guiding magnetic strip laid on the ground for guiding the AGV transport vehicle to travel along a predetermined path, and a power supply providing power for the AGV transport vehicle, usually a rechargeable battery.

[0094] The AGV transport vehicle receives transport instructions from the warehouse control system through the control panel, and the instructions contain the current position and target position of the material. The control panel calculates the optimal path according to the preset map and path planning algorithm. The AGV transport vehicle starts, and the guiding sensor starts to detect the guiding magnetic strip to ensure that the vehicle travels along the predetermined path. The obstacle avoidance sensor monitors obstacles ahead in real time. If there are obstacles, the AGV transport vehicle will automatically stop or bypass to ensure safety. After the AGV transport vehicle reaches the target position, it communicates with the equipment at the target position through the photoelectric control signal sensor to confirm that the material has been correctly placed. After completing the task, the AGV transport vehicle returns to the charging station according to the system instruction and automatically charges to prepare for the next task.

[0095] The AGV transport vehicle can automatically execute transport tasks, reducing the need for manual handling and improving the operation efficiency of the warehouse. Error reduction: Automated transport reduces errors caused by human operation and improves the accuracy and reliability of material transport. Enhanced safety: Safety devices such as obstacle avoidance sensors and status indicator lights ensure the safety of the AGV transport vehicle during operation and reduce the risk of accidents.

[0096] In one embodiment, the scheduling and processing module 140 is configured to receive upper-layer control instructions and distribute the control instructions to the corresponding execution devices, identification devices, and transport equipment.

[0097] Optionally, the scheduling processing module includes an intelligent scheduling algorithm, a task management unit, WCS task workstation distribution, camera image acquisition and recognition, etc. It has an intelligent integrated control platform that can simultaneously control backend devices such as manipulators, PLCs, and cameras and interact with the front-end scheduling system in a timely manner. This platform can also be customized with various demand logics to improve equipment utilization. Specifically, the intelligent scheduling algorithm schedules AGV vehicles to perform handling tasks according to the tasks issued by the planned scheduling system, and realizes the linkage of AGV vehicles with other devices such as robots, gratings, and PLCs during the handling process. The task management unit supervises the task execution situation, records the task completion status, and issues an alarm when an abnormality occurs. It mainly automatically manages the tasks issued by the planned scheduling system, supervises task execution, completion, and abnormal situations. The execution management is carried out in the mode of individual tasks, which is convenient for subsequent search and abnormality troubleshooting of individual problematic materials. The WCS workstation receives the task instructions issued by the scheduling management system based on the Http: / / communication protocol, assigns tasks to the robot and executes them, and logically processes the front-end intelligent human-computer interaction and WCS information. The camera takes pictures to identify the external dimension point clouds of different materials, combines the model point cloud data collected by the algorithm for relative matching, sends the position to the robot, and the robot unpacks and places it on the shelf. The unique 3D plus 2D point cloud image acquisition and recognition method is different from ordinary recognition on the market, with higher efficiency and accuracy.

[0098] For example, in an automated warehouse environment, the upper layer issues a task instruction to the scheduling processing module 140, requiring a batch of materials to be transported from the inbound area to the specified shelf and unpacked and placed. After receiving the instruction, the scheduling processing module 140 optimizes task allocation and path planning through the intelligent scheduling algorithm, and then assigns the tasks to the AGV vehicle, manipulator, and camera.

[0099] AGV vehicle: According to the optimized path planning, the AGV vehicle transports the materials from the inbound area to the specified location and is linked with devices such as robots, gratings, and PLCs to ensure safety and efficiency.

[0100] Camera image acquisition and recognition: The camera takes a 3D depth map and a 2D image of the material, and through the 3D plus 2D point cloud image acquisition and recognition method, identifies the external dimension point cloud of the material and matches it with the pre-collected model point cloud data to determine the position of the material.

[0101] Manipulator operation: The camera sends the position information of the material to the manipulator, and the manipulator unpacks the material according to the position information and places it on the specified shelf.

[0102] Task management unit: Supervises the task execution situation in real time and records the task completion status. If an abnormality occurs, the task management unit will issue an alarm and notify relevant personnel for handling.

[0103] The efficient application of the scheduling processing module 140 in the automated warehouse not only improves the operational efficiency and safety of the warehouse, but also enhances the user experience and system flexibility.

[0104] In one embodiment, the system includes an electric control cabinet, which includes a power cable, a programmable logic controller (PLC), a servo drive, and an encoder. The electric control cabinet is used to monitor and control the operating state of the system.

[0105] Optionally, the power cable provides a stable power supply for the system. The programmable logic controller (PLC), as the core of the control system, is responsible for processing sensor signals, executing control logic, and outputting control signals. The servo drive is used to control the motor of the robotic arm to achieve precise position control and speed control. The encoder is installed on the motor shaft to detect the current position and speed of the motor and feedback this information to the PLC. The sensors include position sensors, torque sensors, temperature sensors, etc., which are used to monitor the working state of the robotic arm and environmental parameters.

[0106] The electric control cabinet not only improves the accuracy and safety of the system, but also enhances the flexibility and maintenance convenience of the system.

[0107] Combined with actual application examples:

[0108] Example 1: Depalletizing and palletizing, the process is as follows:

[0109] 1. Task assignment: After the system assigns a task, the conveyor line sends the goods out of the warehouse to the designated station. When in place, the robotic arm system receives the task.

[0110] 2. Task execution: The robotic arm system issues internal tasks according to the information such as the type, quantity, and size of the materials included in the received task. After the robotic arm receives the instruction and position points, it performs grasping and places the materials at the designated positions, repeating the task execution until the task is completed.

[0111] 3. Task end: After the task is completed, the robotic arm returns to its original position to wait for the next task. The empty pallet on the conveyor line returns, and the pallet with goods is transported away by the AGV.

[0112] In example 1 of depalletizing and palletizing, through the cooperation of the automated robotic arm and the vision system, the handling and stacking efficiency of the goods is improved, manual intervention is reduced, and the overall operation speed is increased. The precise positioning of the vision camera and the high-precision operation of the robotic arm reduce errors caused by manual operation. The PLC records key data, which is convenient for subsequent data analysis and fault troubleshooting, improving the reliability and stability of the system.

[0113] Example 2: Picking station, the process is as follows:

[0114] 1. Task assignment: After the system assigns a task, the conveyor line will send the incoming material basket with goods out of the warehouse to the designated station. After it arrives, the robotic arm system receives the task.

[0115] 2. Task execution: The robotic arm system issues internal tasks according to the information such as the type, quantity, and size of the materials included in the received task. The vision camera takes pictures for positioning. After obtaining the position of the materials, the robotic arm receives the instructions and points and then executes the grasping and places them in the discharging basket. The task execution is repeated until the task is completed.

[0116] 3. Task completion: After the task is completed, the robotic arm returns to its original position to wait for the next task. The conveyor line returns the incoming material basket and transports the discharging basket to the shipping outlet.

[0117] Example 2: The picking station, through the precise positioning of the vision camera and the rapid grasping of the robotic arm, has greatly improved the picking efficiency and reduced the error rate. The automated operation reduces the chance of manual contact with dangerous goods and improves the safety of the operation. The system records the data of each picking, which is convenient for optimizing the picking strategy and improving the overall efficiency.

[0118] In summary, an intelligent outbound storage system for a robotic three-dimensional warehouse provided by an embodiment of the present application includes an execution device for replacing a tool corresponding to a task instruction according to the task instruction and performing an operation corresponding to the task instruction through the tool; an identification device for obtaining a 3D image by photographing a box through a 3D camera and obtaining color information by photographing through a 2D camera, and converting the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determining the position, size, and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image; a transportation device for transporting the material to a designated position according to the received transportation instruction; and a scheduling and processing module for receiving an upper-layer control instruction and distributing the control instruction to the corresponding execution device, identification device, and transportation device. It realizes a great improvement in the efficiency of the military product warehouse for inbound and outbound, completes the automation of the inbound and outbound of the vault, reduces the labor cost, and increases the accuracy. It solves the problem of low efficiency in the intelligent outbound storage system of the robotic three-dimensional warehouse in the related art.

[0119] In a second aspect, an embodiment of the present application provides an intelligent outbound storage method for a robotic three-dimensional warehouse. Figure 2 It is a flowchart of an intelligent outbound storage of a robotic three-dimensional warehouse, as Figure 2 shown, and is applied to the above intelligent outbound storage system for a robotic three-dimensional warehouse. An intelligent outbound storage method for a robotic three-dimensional warehouse includes:

[0120] Step S101: Receive an outbound task instruction from the upper layer control, and the task instruction includes the required material information.

[0121] Step S102: Capture the material using a 3D camera to obtain a 3D image, and capture the material using a 2D camera to obtain color information; convert the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determine the position, size, and color information of the material based on the three-dimensional coordinates of the 3D image and the color information of the 2D image.

[0122] Step S103: Receive and parse the task instructions through the scheduling and processing module, and allocate corresponding tasks to the execution device, identification device, and transportation equipment according to the task instructions.

[0123] Step S104: According to the task instructions, the execution device switches to the tool corresponding to the task instructions, and performs operations such as grasping, transporting, and placing the material through the tool.

[0124] Step S105: The transportation equipment transports the material from the storage location to the designated location according to the received transportation instructions.

[0125] Step S106: After the material is successfully transported to the designated location, confirm that the task is completed, and record and update the system status.

[0126] In summary, a method for intelligent outbound of a robot three-dimensional warehouse provided by the present application. An execution device is used to replace the tool corresponding to the task instructions according to the task instructions, and perform operations corresponding to the task instructions through the tool; an identification device is used to capture the box using a 3D camera to obtain a 3D image, and capture color information using a 2D camera, and convert the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determine the position, size, and color information of the material based on the three-dimensional coordinates of the 3D image and the color information of the 2D image; a transportation equipment is used to transport the material to the designated location according to the received transportation instructions; a scheduling and processing module is used to receive the upper-layer control instructions and allocate the control instructions to the corresponding execution device, identification device, and transportation equipment. It greatly improves the efficiency of the military product warehouse during inbound and outbound, realizes the automation of inbound and outbound of the vault, reduces the labor cost, and increases the accuracy. Solve the problem of low efficiency in the intelligent outbound system of the robot three-dimensional warehouse in the related technology.

[0127] It should be noted that a system for intelligent outbound of a robot three-dimensional warehouse provided in this embodiment is used to implement the above-mentioned implementation manners, and those that have been described will not be repeated. As used above, terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the above embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0128] In a third aspect, an embodiment of the present application provides an electronic device, Figure 3 which is a block diagram of an electronic device shown according to an exemplary embodiment. AsFigure 3 As shown, the electronic device may include a processor 31 and a memory 32 storing computer program instructions.

[0129] Specifically, the processor 31 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0130] Among them, the memory 32 may include a mass storage for data or instructions. By way of example and not limitation, the memory 32 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 32 may include removable or non-removable (or fixed) media. Where appropriate, the memory 32 may be internal or external to the data processing device. In a particular embodiment, the memory 32 is non-volatile memory. In a particular embodiment, the memory 32 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. Where appropriate, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended date out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0131] The memory 32 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 31.

[0132] The processor 31 reads and executes the computer program instructions stored in the memory 32 to implement any one of the robot stereo library intelligent outbound methods in the above embodiments.

[0133] In one embodiment, a device for intelligent outbound of a robot stereo library may further include a communication interface 33 and a bus 30. Among them, as Figure 3 shown, the processor 31, the memory 32, and the communication interface 33 are connected through the bus 30 to complete communication with each other.

[0134] The communication interface 33 is used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application. The communication port 33 can also implement data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0135] Bus 30 includes hardware, software, or both, and couples components of a device for intelligent outbound of a robotic stereoscopic library to each other. Bus 30 includes, but is not limited to, at least one of the following: Data Bus, Address Bus, Control Bus, Expansion Bus, Local Bus. By way of example and not limitation, Bus 30 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, Bus 30 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0136] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, a method for intelligent outbound of a robotic stereoscopic library provided in the first aspect is implemented.

[0137] Among them, the more specific forms that the readable storage medium may adopt may include, but are not limited to: portable disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0138] In a possible implementation manner, the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps of implementing an intelligent outbound method for a robot three-dimensional library provided in the first aspect.

[0139] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, executed as an independent software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0140] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0141] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An intelligent outbound system for a robotic stereoscopic warehouse, characterized in that, The system includes an execution device, an identification device, a transportation device, and a scheduling and processing module; wherein, The execution device is configured to replace the tool corresponding to the task instruction according to the task instruction, and perform an operation corresponding to the task instruction through the tool; The identification device is configured to capture a box by a 3D camera to obtain a 3D image, and capture color information by a 2D camera, and convert the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determine the position, size, and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image; The transportation device is configured to transport the material to a specified position according to the received transportation instruction; The scheduling and processing module is configured to receive an upper-layer control instruction, and distribute the control instruction to the corresponding execution device, identification device, and transportation device.

2. The system according to claim 1, wherein When the identification device obtains the size and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image, it is configured to: Define a region of interest including the material to be identified in the 3D image, and extract the point cloud data within the region of interest; Generate a planar mask for marking the point cloud data within the region of interest according to the point cloud data; Use the planar mask to convert the extracted point cloud data into three-dimensional coordinates in the world coordinate system; In the three-dimensional coordinates, identify the shape of the target material that meets the predetermined parameters through a shape detection algorithm; Convert the shape of the target material to the 2D image through the internal and external parameters of the 3D camera, and combine the color information of the 2D image, and use image processing technology to classify the color of the shape of the target material to determine the color of the shape of the target material; Based on the shape of the target material and the color of the shape of the target material, determine the position, size, and color information of the shape of the target material.

3. The system according to claim 1, wherein When the identification device converts the depth map in the 3D image into three-dimensional coordinates in the world coordinate system, it is configured to: Obtain the internal and external parameters of the 3D camera by calibrating the 3D camera; Use the 3D camera to obtain the depth map of the real scene, and convert the pixel coordinates of the depth map into three-dimensional coordinates in the world coordinate system through the internal and external parameters; Calculate the actual distance of the center point of the world coordinate system and the depth value of the center point of the depth map, calculate the ratio of the actual distance and the depth value, and obtain the conversion ratio between the depth image pixels and the real actual measurement based on the ratio; Based on the conversion ratio, obtain the actual length of the material in the real world through the pixel distance of the depth map.

4. The system according to claim 1, characterized in that, The execution device includes at least one manipulator, and the manipulator replaces the tool corresponding to the task instruction according to the task instruction, and performs operations of grasping, transporting, and placing materials; wherein, The tool includes a fixture and a suction tool, and is used for grasping different types of materials.

5. The system according to claim 1, characterized in that, The transportation device includes an AGV transport vehicle, and the AGV transport vehicle is used for automatically transporting materials in the warehouse according to the task instruction; wherein, The AGV transport vehicle includes a control panel, a guiding sensor, a direction potentiometer, a status indicator light, an obstacle avoidance sensor, an optoelectronic control signal sensor, a driving unit, a guiding magnetic strip, and a power supply.

6. The system according to claim 1, characterized in that, The scheduling and processing module further includes a task management unit, which supervises the execution of tasks, records the task completion status, and issues an alarm when an abnormality occurs.

7. The system according to claim 1, characterized in that The system includes an electric control cabinet, which includes a power cable, a programmable logic controller (PLC), a servo driver, and an encoder. The electric control cabinet is used to monitor and control the operating status of the system.

8. An intelligent outbound method for a robotic three-dimensional warehouse, characterized in that, The method is applied to the system according to any one of claims 1-7, and the method includes: Receiving an outbound task instruction from the upper-level control, where the task instruction includes the required material information; Taking a 3D image of the material through the 3D camera and obtaining color information by taking a 2D image of the material; converting the depth map in the 3D image into three-dimensional coordinates in the world coordinate system; determining the position, size, and color information of the material according to the three-dimensional coordinates of the 3D image and the color information of the 2D image; Receiving and parsing the task instruction through the scheduling and processing module, and allocating corresponding tasks to the execution device, the identification device, and the transportation device according to the task instruction; According to the task instruction, the execution device switches to the tool corresponding to the task instruction, and performs operations of grasping, transporting, and placing the material through the tool; The transportation device transports the material from the storage location to the designated location according to the received transportation instruction; After the material is successfully transported to the designated location, confirm that the task is completed, and record and update the system status.

9. An electronic device, characterized in that, It includes a memory and a processor, a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements a robot three-dimensional library intelligent outbound system according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements a robot three-dimensional library intelligent outbound system according to any one of claims 1 to 7.

Images