Adaptive carrying method of robot, carrying robot, storage medium and system
By receiving task instructions, selecting observation points, acquiring images, and analyzing bin placement information, the robot achieves adaptive handling of bins in various scenarios, solving the problem of application scenario limitations in existing technologies and improving the applicability of handling robots.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING PHOENIX TECHNOLOGY CO LTD
- Filing Date
- 2026-03-27
- Publication Date
- 2026-07-14
Smart Images
Figure CN122378680A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of robotics, and more specifically, to an adaptive handling method for a robot, a handling robot, a storage medium, and a system. Background Technology
[0002] With the development of artificial intelligence technology, material handling robots have gradually begun to be used in industrial settings. Some existing material handling robots, such as wheeled robots and automated guided vehicles (AGVs), are already being used in warehousing and logistics scenarios.
[0003] However, existing handling robots can generally only identify objects with fixed specifications and fixed placement patterns, such as identifying and moving boxes of a fixed style. Therefore, the application scenarios of existing handling robots are quite limited. Summary of the Invention
[0004] The purpose of this application is to address the shortcomings of the prior art by providing an adaptive handling method, handling robot, storage medium, and system for robots, thereby solving the problem of limited applicability of existing handling robots in handling processes.
[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, one embodiment of this application provides an adaptive handling method for a robot, the method comprising: Receive a handling task instruction, the handling task instruction including: the number of boxes to be handled; According to the handling task instruction, a target observation point is selected from multiple preset observation points using a preset algorithm. The preset observation point is a designated location in the material box area. Control the robot to move to the target observation point and acquire the image to be processed; Based on the image to be processed and the image analysis algorithm, the placement information of the material box to be transported is determined from the material box area in the image to be processed; Based on the placement information of the boxes to be transported, a transport command is generated to control the robot to transport the boxes to the target location.
[0006] Optionally, determining the placement information of the bins to be transported from the bin area in the image to be processed based on the image to be processed and the image analysis algorithm includes: Based on the image to be processed and the image analysis algorithm, at least one bin and its layout information are identified from the image to be processed. Based on the layout information of the bins and the handling logic, the bins to be handled and their placement information are determined, wherein the placement information includes coordinate information.
[0007] Optionally, determining the material box to be transported and its placement information based on the layout information of the material box and the handling logic includes: Based on the layout information of the bins and the handling logic, the depth layering of the bin area is identified from the depth direction, the longitudinal layering of the bin area is identified from the longitudinal direction, and the obstruction relationship between adjacent bins is identified to obtain the bin grabbing order. Based on the grabbing order of the bins, determine the bins to be transported and their placement information.
[0008] Optionally, after identifying at least one bin and its layout information from the image to be processed based on the image to be processed and the image analysis algorithm, the method further includes: Obtain the center point coordinates of each of the aforementioned bins; Based on the center point coordinates of each of the material bins, obtain the current distance between each material bin and the robot; The validity of each bin is determined based on the current distance between each bin and the robot, and a preset threshold. If no valid bin is found, the system is marked as failing and the robot is moved to the next observation point.
[0009] Optionally, the method further includes: If a valid bin exists, the width information of this side of the bin is obtained according to the direction of the bin area facing the target observation point; Determine whether there is a valid long side on this side based on the width information; If there is no valid long side, the operation is marked as a failure and the robot is controlled to move to the next observation point.
[0010] Optionally, after determining whether there is a valid long side on this side based on the width information, the method further includes: If a valid long side exists, retain the bin data corresponding to the valid long side and delete the data of other bins.
[0011] Optionally, the step of generating a handling command based on the placement information of the bins to be handled to control the robot to move the bins to the target location includes: Based on the placement information of the material box to be transported, calculate and obtain the coordinate information of the center point of the long side end face of the material box to be transported; Based on the coordinate information of the center point of the long side end face of the material box to be transported, calculate the navigation point information of the material box to be transported; Based on the navigation point information, the robot is controlled to move to the navigation point, and then a handling instruction is generated according to a preset vision algorithm to control the robot to move the material box to be handled to the target location.
[0012] Secondly, another embodiment of this application provides a handling robot, which includes a processor and a memory. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor executes the machine-readable instructions to perform the adaptive handling method of the robot described above.
[0013] Thirdly, another embodiment of this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the aforementioned adaptive handling method of the robot.
[0014] Fourthly, another embodiment of this application provides an adaptive handling system for a robot, including a cloud server and the handling robot described in the second aspect, wherein the cloud server is used to send handling task instructions to the handling robot.
[0015] The beneficial effects of this application are as follows: the robot selects a target observation point from multiple preset observation points according to the handling task instructions, controls the robot to move to the target observation point and acquires the image to be processed, and determines the placement information of the material box to be transported from the material box area in the material box area of the image to be processed according to the image to be processed, and then generates a handling instruction to control the robot to transport the material box to be transported to the target position. This realizes the recognition of material box areas in various scenarios, no longer restricts the placement of material boxes, and flexibly controls the robot to complete the handling according to the recognized information, so that the handling robot can be applied to more scenarios. Attached Figure Description
[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This application provides an architectural diagram of an adaptive handling system for a robot. Figure 2 This is a schematic flowchart of an adaptive handling method for a robot provided in an embodiment of this application; Figure 3 This is a schematic flowchart of an adaptive handling method for a robot provided in another embodiment of this application; Figure 4 This is a schematic diagram of the structure of an adaptive handling device for a robot provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a handling robot provided in an embodiment of this application. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0019] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0020] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.
[0021] Figure 1 This is a schematic diagram of the architecture of an adaptive handling system for a robot provided in an embodiment of this application, as shown below. Figure 1 As shown, the system includes a cloud server 01 and a transport robot 02. The cloud server 01 is used to send transport task instructions to the transport robot 02.
[0022] Optionally, the cloud server 01 can communicate with the control terminal to receive control commands from the control terminal and then send handling task commands to the handling robot 02. Of course, in some scenarios, the cloud server 01 can also send handling task commands to the handling robot 02 according to preset handling rules, which is not limited here. Alternatively, the handling robot 02 can also directly receive handling task commands sent by the staff through the control terminal.
[0023] like Figure 1 As shown, the handling robot 02 can move one or more boxes in the box area 03 according to the handling task instruction. Specifically, the handling task instruction can instruct the handling robot 02 to move the box to the target location and place it.
[0024] In this embodiment, the transport robot 02 can integrate a transport robotic arm and an image acquisition camera. Furthermore, the transport robot 02 can be a wheeled robot, i.e., it includes small movable wheels at its bottom to drive its movement; of course, it can also be other forms of transport robots, which are not limited here.
[0025] The handling robot 02 in this embodiment uses a new adaptive handling method and image analysis to flexibly handle boxes of different sizes and placements.
[0026] Figure 2 This is a schematic flowchart of an adaptive handling method for a robot provided in an embodiment of this application. The method is executed by the aforementioned handling robot and includes: S201, Receive handling task instructions.
[0027] Specifically, in this embodiment, the task may involve receiving a handling instruction from a cloud server or control terminal (such as a mobile phone, remote control, etc.). This handling instruction may include the number of boxes to be handled.
[0028] S202. According to the transport task instruction, select the target observation point from multiple preset observation points using a preset algorithm.
[0029] The material bin area is where material bins are stacked. The bins can be stacked in multiple layers, with multiple bins per layer. This embodiment does not restrict the number of bins, the number of layers, or the arrangement of the bins. Multiple preset observation points can be designated locations within the material bin area. Optionally, there can generally be four preset observation points: the rear, front, and sides of the material bin area (which can be the right and left sides from the perspective of the handling robot), without limitation.
[0030] The preset algorithm can be a polling algorithm, where several preset observation points are used as target observation points in turn according to certain rules, or it can be a preset observation point that is randomly selected. For example, the target observation points can be used in turn in the order of rear, right, front, and left, and the observation robot can move in sequence.
[0031] S203. Control the robot to move to the above-mentioned target observation point and acquire the image to be processed.
[0032] Optionally, the robot can integrate an RGB-D depth camera, which can be integrated into the head of the transport robot. After the robot moves to the target observation point, the depth camera can acquire RGB and depth images in real time. Specifically, the robot can record the identifier of the target observation point, for example, by including it in task data, and then record the corresponding image to be processed.
[0033] In the specific implementation process, after the robot reaches the target observation point, the robot's state machine can switch to the analysis state (TOPOLOGY_ANALYSIS_STEP2). The robot's software system can obtain the collected RGB image and depth image through the subscription mechanism. Then, it is necessary to verify the timestamps of the RGB image and depth image to ensure that the time difference between the two images is within a preset threshold (e.g., within 10 milliseconds). After the time synchronization verification is completed, the corresponding RGB image and depth image are confirmed as copies for backup for subsequent analysis.
[0034] S204. Based on the image to be processed and the image analysis algorithm, determine the placement information of the material box to be transported from the material box area in the image to be processed.
[0035] After the state machine completes image time synchronization, the robot's software system can further call the perception service module to send the RGB image and depth image to the perception service module. The perception service module can analyze the image to be processed to obtain the placement information of the bins in the bin area. This placement information can indicate the placement of the bins, such as coordinate information, angle information, layer information, size information, etc. Specifically, it can include dimensions and coordinates in different directions, such as the width and height of the bins, the coordinates of the center point in the robot coordinate system, the number of bins in each column, etc.
[0036] S205. Based on the placement information of the boxes to be transported, generate a transport command to control the robot to transport the boxes to the target location.
[0037] Once the placement information of the boxes to be moved is determined, the number of boxes to be moved in the handling task instruction and the order in which the boxes are placed can be used to determine the boxes to be moved. Based on the placement information of the boxes to be moved, the robot's movement trajectory and handling position can be determined, and a handling instruction can be generated. The robot moves to the handling position according to the handling instruction, uses a robotic arm to pick up the boxes to be moved, and then moves according to the target position to place the boxes to be moved into the target position to complete the handling task.
[0038] Optionally, the handling instructions need to include instructing the robot to move to the handling position (navigation point). This can be based on the center coordinates (x, y, z) of the bin to be handled as the position the robotic arm's end effector needs to reach (grip point position), thereby calculating the robot's motion trajectory. It also includes the gripping posture (the direction the robotic arm's end effector approaches the bin, typically represented by quaternions or Euler angles (roll, pitch, yaw) to ensure the gripper is correctly aligned with the bin's gripping surface). After gripping, the coordinates and posture of the placement point must be determined based on the target position's coordinates. The bin's center coordinates must be placed at the placement point. The coordinates and posture of this placement point must also be calculated based on the previous bin's placement point and the stacking logic to ensure the bins can meet stacking requirements.
[0039] In this embodiment, the robot selects a target observation point from multiple preset observation points according to the handling task instruction, controls the robot to move to the target observation point and acquires the image to be processed. Based on the image to be processed and the image analysis algorithm, the placement information of the material box to be transported is determined from the material box area in the image to be processed, and then a handling instruction is generated to control the robot to transport the material box to the target position. This realizes the recognition of material box areas in various scenarios, no longer restricting the placement of the material box, and flexibly controlling the robot to complete the handling based on the recognized information, so that the handling robot can be applied to more scenarios.
[0040] Optionally, the process of determining the coordinate and size information of the bin to be transported from the bin region in the image to be processed, based on the image to be processed and the image analysis algorithm, may include: identifying at least one bin and its layout information from the image to be processed, and then determining the bin to be transported and its placement information based on the bin layout information and transport logic. In this embodiment, the placement information may include coordinate information, but is not limited to this; for example, it may be the coordinate information of the center point of the bin.
[0041] Specifically, a deep learning model can be used to identify at least one bin and its layout information from the image to be processed. For example, the perception service module can use a deep learning model to analyze the acquired images. This deep learning model can be trained on a large number of sample images, which may include multiple bins and are labeled with the placement information of each bin. The acquired RGB image and depth image are input into the deep learning model, that is, the deep learning model combines two-dimensional and three-dimensional images to segment the bin area in the image to be processed and obtain the bin placement information. Optionally, multiple bins in the bin area can be identified, and the arrangement of these bins can be identified, such as the number of columns and rows of bins, the width and height of each bin, the coordinates of the center point, and the number of bins in each column. Specifically, the perception service module can store the identified information in the "back_perception_result" dictionary, but is not limited to this.
[0042] After obtaining the layout information of the bins, the material bins can be moved based on the handling logic, such as moving from top to bottom or from outside to inside. Then, combined with the number of bins to be moved, the bins can be determined as the bins to be moved. Furthermore, the placement information of the bins to be moved can be determined based on the placement information obtained by the perception service module.
[0043] Furthermore, the above-mentioned determination of the material bins to be transported and their placement information based on the layout information and handling logic may specifically include: Based on the layout information of the bins and the handling logic, the depth layering of the bin area is identified from the depth direction, the longitudinal layering of the bin area is identified from the vertical direction, and the obstruction relationship between adjacent bins is identified to obtain the bin grabbing order; based on the bin grabbing order, the bins to be transported and their placement information are determined.
[0044] First, image analysis algorithms (deep learning models) are used to analyze the bin areas in the image to be processed, that is, areas where multiple bins are stacked. Then, the layering, sorting, and neighbor blocking relationships (short_layout_block) of the bins are identified. The perception service module can first perform depth layering, grouping the bins according to the depth value of the depth image, with depth=0 as the front layer and depth=1 as the back layer. In the created record dictionary (e.g., the layers dictionary), the layering result is recorded with depth as the key and the center point coordinates (x, y, z) of the bin in that layer as the value.
[0045] Furthermore, based on the depth and vertical layering of the bins, the obstruction relationship between adjacent bins, and the bin handling logic, the gripping order (i.e., gripping priority) of the bins can be determined. The handling logic (front layer priority, upper-middle vertical layer priority, outer layer priority) can include the overall gripping strategy to facilitate better bin handling, ensuring that the handling process does not affect other bins and is easy for the robot's robotic arm to operate. 1. Sort by depth from smallest to largest (first layer first); 2. Within the same depth, sort by coordinate z from largest to smallest (prioritizing the upper and middle layers vertically). In some scenarios, after traversing the bins sorted by z, if the vertical drop is less than a preset threshold, they are classified into the same physical layer; otherwise, they are recorded as a new physical layer.
[0046] Optionally, in some scenarios where each box has the same length, width, and height, the aforementioned handling task instruction can also specify the preset dimensions of the boxes. The preset dimensions are compared with the depth information identified by the robot, and the difference between the center point coordinates and the bottom edge coordinates of the boxes. If the height difference between adjacent boxes is less than a preset threshold, then the two adjacent boxes are considered to be on the same physical layer. Optionally, if the boxes are not of the preset dimensions, the difference between the center point coordinates and the bottom edge coordinates of the boxes can also be used to compare whether the height difference between adjacent boxes is less than a preset threshold to determine whether they belong to the same physical layer; this is not specifically limited here.
[0047] 3. For the same z-height, sort by |y| from largest to smallest (outer edge takes priority).
[0048] The criteria for determining the obstruction relationship between adjacent bins can include: belonging to the same physical layer, overlapping in a specified direction (x and y directions), meaning that adjacent bins have overlapping areas along their wide or long sides, and the width of the overlapping area is greater than a preset width (e.g., 0.05 meters, 0.03 meters, etc.). When determining the grabbing order of the bins to be moved, bins without obstruction should be moved first. For example, for a target observation point, if there is an obstruction relationship in one direction but not in others, then the bins should be moved from those other directions.
[0049] Optionally, when recording, the following can be recorded for each column of bins: all_longside (long side direction vector), all_positions (position coordinates), all_centers (center point), all_observation_point (observation point), and all_col_boxes (number of bins).
[0050] Figure 3The flowchart of an adaptive handling method for a robot provided in another embodiment of this application is shown. During the analysis process of the above-mentioned perception service module, a data filtering and verification process may also be included to determine whether the identified data is valid data. If it is invalid data, it may be necessary to trigger the movement to the next observation point (new target observation point) and repeat the above method.
[0051] Taking the robot at the target observation point as a reference, there are two invalid determination conditions: one is the distance between the center point of each bin and the current robot, and the other is whether the bin's long side is facing the robot.
[0052] Optionally, after identifying at least one bin and its layout information from the image to be processed, the method includes: S301. Obtain the center point coordinates of each bin.
[0053] S302. Based on the center point coordinates of each bin, obtain the current distance between each bin and the robot.
[0054] Optionally, the horizontal distance between each bin and the robot can be calculated using Euclidean distance based on the center point coordinates of each bin.
[0055] S303. Determine the validity of each bin based on the current distance between each bin and the robot, and a preset threshold.
[0056] If the current distance between the bin and the robot is greater than a preset threshold, the bin's data is considered invalid. Consequently, all relevant data about the bin, such as center point coordinates, width, height, and corresponding indexes, are deleted from the records.
[0057] By repeating this process, we can analyze whether the distance between each bin and the robot is less than or equal to a preset threshold. If it is determined that all bins acquired at the current observation point are invalid, it means that the perception result at that observation point is a failure.
[0058] S304. If no valid bin exists, mark the failure and control the robot to move to the next observation point.
[0059] The robot moves to the next observation point and continues to collect and process images, and determines the placement information of the boxes to be transported from the box area in the image based on the images and image analysis algorithms.
[0060] Further optionally, it may also include: S305. If a valid bin exists, the width information of the bin on this side is obtained according to the direction of the bin area facing the target observation point.
[0061] That is, to determine whether each bin on this side is along the long side, with the target observation point as a reference.
[0062] S306. Determine whether there is a valid long side on this side based on the width information mentioned above.
[0063] Optionally, if the width information is greater than the preset length value, the bin is determined to be a valid long side, i.e., the long side faces the robot. If the width information is less than or equal to the preset length value, the short side of the bin faces the robot and is recorded as an invalid long side. The relevant data and indexes corresponding to the bin are deleted. That is, the valid long side data and the corresponding bin are retained in the perception result, while other bin data are deleted.
[0064] S307. If there is no valid long side, mark it as a failure and control the robot to move to the next observation point.
[0065] The system continues until each observation point has been traversed, and for each observation point, the system performs the following steps: acquiring images to be processed, determining the handling information of the material box to be transported from the material box area in the image to be processed based on the image to be processed and the image analysis algorithm. The perception service module then completes the task and reports it to the robot.
[0066] Optionally, if a valid long side exists, the bin data corresponding to the valid long side is retained, and the data of other bins is deleted.
[0067] It should be noted that the embodiments of this application are for determining the navigation point based on the long side of the material box during the robot's handling process. In the above embodiments, after determining the layout information and grasping order of the material box, the robot's navigation point, i.e., the target position, is calculated based on the center point of the long side of the material box.
[0068] Optionally, the robot's long side midpoint calculation method can be used. Based on the long side direction vector of the bin, the safety offset is dynamically calculated, and then the center point of the long side end face of the bin to be transported that the robot needs to align with is calculated. Then, based on the center point of the long side end face of the bin to be transported, the relative navigation point of the robot chassis is calculated. Then, combined with the absolute position of the target observation point, the final target position in the navigation map coordinate system is calculated through coordinate transformation, which is the final transport point (absolute navigation point).
[0069] Accordingly, the above-mentioned method of generating a handling instruction based on the placement information of the material box to be handled to control the robot to move the material box to the target position may include: calculating and obtaining the coordinate information of the center point of the long side end face of the material box to be handled based on the placement information of the material box to be handled; calculating the navigation point information of the material box to be handled based on the coordinate information of the center point of the long side end face of the material box to be handled; controlling the robot to move to the navigation point based on the navigation point information; and then generating a handling instruction based on a preset vision algorithm to control the robot to move the material box to the target position.
[0070] After navigating to the final handling point, the robot activates the perception service module to sense and locate the center coordinates of the short side of the bin to be handled using visual algorithms, then performs the grasping action. It's important to note that in adaptive handling, the bin may be placed arbitrarily. The long side of the bin is stable, facilitating the determination of the navigation point. Grasping from the short side is more consistent with the robotic arm's movements, resulting in a more stable grasp. The robot can more accurately calculate the bin's deflection angle relative to itself, and the adaptive control algorithm adjusts the posture of the robotic arm or the robot body accordingly to achieve "centering" and stable bin grasping. Furthermore, the distance between the bin and the robot is considered, primarily based on the robotic arm length to determine the aforementioned preset threshold. If the distance is too great, the robot arm may not be able to grasp the bin deeply enough, potentially leading to problems such as unstable grip or failure to grasp.
[0071] Reference Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of an adaptive handling device for a robot provided in an embodiment of this application. The device includes: The receiving module 401 is used to receive a handling task instruction, which includes: the number of boxes to be handled; The calculation module 402 is used to select a target observation point from multiple preset observation points according to the handling task instruction using a preset algorithm, wherein the preset observation point is a specified location in the material box area; Control module 403 is used to control the robot to move to the target observation point and acquire the image to be processed; Analysis module 404 is used to determine the placement information of the material box to be transported from the material box area in the image to be processed based on the image to be processed and the image analysis algorithm; The handling module 405 is used to generate handling instructions based on the placement information of the material box to be handled, and control the robot to handle the material box to be handled to the target position.
[0072] Optionally, the analysis module 404 is specifically used to identify at least one bin and its layout information from the image to be processed based on the image to be processed and the image analysis algorithm; and to determine the bin to be transported and its placement information based on the layout information and the transport logic, wherein the placement information includes coordinate information.
[0073] Optionally, the analysis module 404 is specifically used to identify the depth layering of the material box area from the depth direction, the longitudinal layering of the material box area from the longitudinal direction, and the blocking relationship between adjacent material boxes based on the layout information of the material box and the handling logic, and to obtain the grasping order of the material boxes. Based on the grabbing order of the bins, determine the bins to be transported and their placement information.
[0074] Optionally, the analysis module 404 is further configured to obtain the center point coordinates of each of the material boxes; obtain the current distance between each material box and the robot based on the center point coordinates of each material box; determine the validity of the material box based on the current distance between each material box and the robot and a preset threshold; if there is no valid material box, mark it as a failure and control the robot to move to the next observation point.
[0075] Optionally, the analysis module 404 is further configured to, if a valid bin exists, obtain the width information of the bin itself based on the direction of the bin area facing the target observation point; determine whether a valid long side exists on this side based on the width information; if no valid long side exists, mark the failure and control the robot to move to the next observation point.
[0076] Optionally, the analysis module 404 is also used to retain the bin data corresponding to the valid long side and delete the data of other bins if a valid long side exists.
[0077] Optionally, the handling module 405 is specifically used to calculate and obtain the coordinate information of the center point of the long side end face of the material box to be handled based on the placement information of the material box to be handled; calculate the navigation point information of the material box to be handled based on the coordinate information of the center point of the long side end face of the material box to be handled; control the robot to move to the navigation point based on the navigation point information; and then generate a handling instruction based on a preset vision algorithm to control the robot to handle the material box to be handled to the target position.
[0078] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.
[0079] This application also provides a handling robot, such as... Figure 5The diagram shown is a structural schematic of a handling robot provided in an embodiment of this application, including: a processor 501 and a memory 502. Optionally, it may also include a bus 503. The memory 502 stores machine-readable instructions executable by the processor 501. The processor 501 and the memory 502 communicate via the bus 503. The processor 501 executes the machine-readable instructions to perform the adaptive handling method of the robot described above.
[0080] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the aforementioned adaptive handling method of the robot.
[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.
[0082] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.
[0083] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. An adaptive handling method for a robot, characterized in that, include: Receive a handling task instruction, the handling task instruction including: the number of boxes to be handled; According to the handling task instruction, a target observation point is selected from multiple preset observation points using a preset algorithm. The preset observation point is a designated location in the material box area. Control the robot to move to the target observation point and acquire the image to be processed; Based on the image to be processed and the image analysis algorithm, the placement information of the material box to be transported is determined from the material box area in the image to be processed; Based on the placement information of the boxes to be transported, a transport command is generated to control the robot to transport the boxes to the target location.
2. The method according to claim 1, characterized in that, The step of determining the placement information of the material bins to be transported from the material bin area in the image to be processed based on the image to be processed and the image analysis algorithm includes: Based on the image to be processed and the image analysis algorithm, at least one bin and its layout information are identified from the image to be processed. Based on the layout information of the bins and the handling logic, the bins to be handled and their placement information are determined, wherein the placement information includes coordinate information.
3. The method according to claim 2, characterized in that, The step of determining the material box to be transported and its placement information based on the layout information of the material box and the handling logic includes: Based on the layout information of the bins and the handling logic, the depth layering of the bin area is identified from the depth direction, the longitudinal layering of the bin area is identified from the longitudinal direction, and the obstruction relationship between adjacent bins is identified to obtain the bin grabbing order. Based on the grabbing order of the bins, determine the bins to be transported and their placement information.
4. The method according to claim 2, characterized in that, After identifying at least one bin and its layout information from the image to be processed based on the image and the image analysis algorithm, the method further includes: Obtain the center point coordinates of each of the aforementioned bins; Based on the center point coordinates of each of the material bins, obtain the current distance between each material bin and the robot; The validity of each bin is determined based on the current distance between each bin and the robot, and a preset threshold. If no valid bin is found, the system is marked as failing and the robot is moved to the next observation point.
5. The method according to claim 4, characterized in that, The method further includes: If a valid bin exists, the width information of this side of the bin is obtained according to the direction of the bin area facing the target observation point; Determine whether there is a valid long side on this side based on the width information; If there is no valid long side, the operation is marked as a failure and the robot is controlled to move to the next observation point.
6. The method according to claim 5, characterized in that, After determining whether there is a valid long side on this side based on the width information, the process further includes: If a valid long side exists, retain the bin data corresponding to the valid long side and delete the data of other bins.
7. The method according to claim 1, characterized in that, The step of generating a handling command based on the placement information of the material box to be handled, controlling the robot to move the material box to the target location, includes: Based on the placement information of the material box to be transported, calculate and obtain the coordinate information of the center point of the long side end face of the material box to be transported; Based on the coordinate information of the center point of the long side end face of the material box to be transported, calculate the navigation point information of the material box to be transported; Based on the navigation point information, the robot is controlled to move to the navigation point, and then a handling instruction is generated according to a preset vision algorithm to control the robot to move the material box to be handled to the target location.
8. A transport robot, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when the electronic device is running, are executed by the processor to perform the adaptive handling method of the robot as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the adaptive handling method of the robot as described in any one of claims 1 to 7.
10. An adaptive handling system for a robot, characterized in that, It includes a cloud server and the handling robot of claim 8, wherein the cloud server is used to send handling task instructions to the handling robot.