Goods delivery robot cabin door control method, goods delivery robot and readable storage medium

Abstract

The invention provides a delivery robot cabin door control method, electronic equipment and a computer readable storage medium. The delivery robot cabin door control method comprises the steps that a target storage plate where a target delivery order of a delivery robot is located is determined, and the delivery robot is provided with a cargo hold comprising a plurality of storage plates and a gripper; acquiring a current placement position of the target delivery order in the target placement plate; controlling the target placement plate to move according to the current placement position, so that the target delivery order is aligned with the grabbing position of the target placement plate; when a goods taking instruction of the target delivery order is received, controlling the gripper to grab the target delivery order from the grabbing position to a cargo hold outlet of the delivery robot; and when the target delivery order is located at the cargo hold exit, controlling the cargo hold door of the delivery robot to be opened. The cargo hold utilization rate of the robot can be improved, and the overall delivery efficiency of the robot is improved.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a delivery robot hatch control method, a delivery robot, and a computer-readable storage medium. Background Art

[0002] As robot technology matures and consumers’ demand for automated services increases, robots are being used in more and more scenarios, such as hotels, retail, warehousing and logistics, etc. Delivery of goods is an important application scenario for robots. For example, hotels usually use robots to deliver daily necessities, meals, takeout, etc. to guest rooms.

[0003] In the related art, in order to improve and ensure delivery safety, the robot is provided with multiple compartments for placing goods, each compartment is used to place an order, and the opening and closing of each compartment door is controlled separately; however, during the actual research and development process, the inventor of the present application found that some orders are smaller and only require a small part of the compartment position, so most of the capacity positions are vacant, resulting in low overall delivery efficiency of the robot. Summary of the invention

[0004] The present application provides a delivery robot hatch control method, a delivery robot, and a computer-readable storage medium, which can improve the utilization rate of the robot's cargo compartment and improve the overall delivery efficiency of the robot.

[0005] In a first aspect, the present application provides a method for controlling a hatch of a delivery robot, wherein the delivery robot is provided with a cargo hold including a plurality of storage plates and a gripper, and the method comprises:

[0006] Determine the target shelf where the target delivery order of the delivery robot is located;

[0007] Obtaining the current placement position of the target delivery order in the target storage board;

[0008] Controlling the target placement plate to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate;

[0009] Upon receiving a pickup instruction for the target delivery order, controlling the gripper to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot;

[0010] When the target delivery order is at the cargo hold exit, the cargo hold door of the delivery robot is controlled to open.

[0011] In some embodiments, the method further comprises:

[0012] After the target storage plate is loaded with goods, an image of the target storage plate after loading with goods is acquired;

[0013] Perform segmentation processing on the image after goods distribution to obtain a segmentation map of each item in the image after goods distribution;

[0014] In response to the order association instruction of each item segmentation diagram, the paired orders of each item segmentation diagram are associated with the target storage board and recorded in the order placement database;

[0015] The step of determining a target storage plate for a target delivery order of the delivery robot includes:

[0016] The storage board associated with the target delivery order is queried from the order placement database to serve as the target storage board of the target delivery order.

[0017] In some embodiments, the method further comprises:

[0018] Associating each item segmentation diagram with a paired order of each item segmentation diagram and recording it in the order placement database;

[0019] After receiving the pickup instruction of the target delivery order, controlling the gripper to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot comprises:

[0020] After receiving the pickup instruction of the target delivery order, obtaining the order comparison chart of the grabbing location;

[0021] Querying the item segmentation map recorded in association with the target delivery order from the order placement database as an order reference map of the target delivery order;

[0022] If the similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot.

[0023] In some embodiments, the associating and recording each item segmentation diagram and the paired order of each item segmentation diagram into the order placement database includes:

[0024] Perform classification processing based on the item segmentation diagrams to obtain item categories of the item segmentation diagrams;

[0025] Obtaining order information of each delivery order of the delivery robot;

[0026] Based on the order information of each delivery order, the item categories of each item segmentation diagram are matched to obtain a preliminary matching order of each item segmentation diagram;

[0027] In response to a confirmation operation of a preliminary pairing order for a first item segmentation diagram among the item segmentation diagrams, the preliminary pairing order for the first item segmentation diagram is associated with the first item segmentation diagram and recorded in an order placement database.

[0028] In some embodiments, the associating and recording each item segmentation diagram and the paired order of each item segmentation diagram into the order placement database includes:

[0029] Perform classification processing based on the item segmentation diagrams to obtain item categories of the item segmentation diagrams;

[0030] Obtaining order information of each delivery order of the delivery robot;

[0031] Based on the order information of each delivery order, the item categories of each item segmentation diagram are matched to obtain a preliminary matching order of each item segmentation diagram;

[0032] In response to a modification operation of the preliminary pairing order of the second item segmentation diagram among the item segmentation diagrams, the modified pairing order of the second item segmentation diagram is associated with the second item segmentation diagram and recorded in the order placement database.

[0033] In some embodiments, obtaining the current placement position of the target delivery order in the target storage board includes:

[0034] photographing the target storage board to obtain a current image of the target storage board;

[0035] Querying the item segmentation map recorded in association with the target delivery order from the order placement database as an order reference map of the target delivery order;

[0036] Matching the current image with the order reference image to obtain an image position of the target delivery order in the current image;

[0037] Based on the image position and a preset image space conversion relationship, a current placement position of the target delivery order in the target storage plate is determined.

[0038] In some embodiments, the method further comprises:

[0039] Based on the post-arrangement image, position recognition is performed on the paired orders of the item segmentation diagrams to obtain order placement positions of the item segmentation diagrams;

[0040] Associating the order placement positions of the item segmentation diagrams with the paired orders of the item segmentation diagrams into the order placement database;

[0041] The obtaining of the current placement position of the target delivery order in the target storage board includes:

[0042] The order placement location recorded in association with the target delivery order is queried from the order placement database to serve as the current placement location of the target delivery order.

[0043] In some embodiments, controlling the target placement plate to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate includes:

[0044] Detecting whether the previous delivery order of the target delivery order has been picked up;

[0045] When it is detected that the pickup of the previous delivery order is completed, the target placement plate is controlled to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate.

[0046] In a second aspect, the present application further provides a delivery robot, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes any one of the delivery robot hatch control methods provided in the present application.

[0047] In a third aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the delivery robot hatch control method.

[0048] In the present application, the target placement board where the target delivery order of the delivery robot is located is controlled to move according to the current placement position of the target delivery order in the target placement board, so that the target delivery order is aligned with the grabbing position of the target placement board; after receiving the pick-up instruction of the target delivery order, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo hold exit of the delivery robot; in this way, when multiple orders are placed in a mixed manner, the target delivery order can be accurately controlled to move to the cargo hold exit, so that multiple orders can be mixed and placed on the same placement board, making full use of the cargo hold storage capacity; when the target delivery order is at the cargo hold exit, the cargo hold door of the delivery robot is controlled to open, so that the picker can pick up the target delivery order to avoid the problem of other orders being picked up by mistake; it can be seen that picking up through this door control method allows orders to be placed in a mixed manner, improves the utilization rate of the robot's cargo hold, and improves the overall delivery efficiency of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 is a schematic structural block diagram of a delivery robot provided in an embodiment of the present application;

[0051] Figure 2 It is a flow chart of a method for controlling a hatch of a delivery robot provided in an embodiment of the present application;

[0052] Figure 3 This is a schematic diagram of a structural framework of a delivery robot in an embodiment of the present application;

[0053] Figure 4 It is a flow chart of a storage board where an order is recorded provided in an embodiment of the present application;

[0054] Figure 5 It is a schematic diagram of an embodiment of step 204 provided in the embodiments of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0056] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0057] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0058] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0059] The embodiment of the present application provides a delivery robot hatch control method, a delivery robot and a computer-readable storage medium. The delivery robot may be a robot used for delivering meals, express delivery, etc.

[0060] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0061] Figure 1 It is a schematic block diagram of the structure of an electronic device provided in an embodiment of the present application.

[0062] like Figure 1 As shown, the electronic device 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected via a bus 103, such as a PCIe (Peripheral Component Interconnect Express) bus.

[0063] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire electronic device 100. The processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0064] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0065] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the embodiment scheme of the present application, and does not constitute a limitation on the electronic device to which the embodiment scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0066] The processor 101 is used to run the computer program stored in the memory 102, and implement any one of the delivery robot hatch control methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is used to run the computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0067] Determine a target storage plate where a target delivery order of the delivery robot is located; obtain a current placement position of the target delivery order on the target storage plate; control the movement of the target storage plate according to the current placement position so that the target delivery order is aligned with a grabbing position of the target storage plate; upon receiving a pickup instruction for the target delivery order, control the gripper to grab the target delivery order from the grabbing position to an exit of a cargo hold of the delivery robot; and when the target delivery order is at the exit of the cargo hold, control the cargo hold door of the delivery robot to open.

[0068] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0069] After the target storage plate is matched, a matched image of the target storage plate is obtained; segmentation processing is performed on the matched image to obtain segmentation diagrams of each item in the matched image; in response to order association instructions of the segmentation diagrams of each item, the paired orders of the segmentation diagrams of each item are associated with the target storage plate and recorded in an order placement database; from the order placement database, the storage plate associated with the target delivery order is queried to serve as the target storage plate of the target delivery order.

[0070] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0071] The item segmentation diagrams are associated with the paired orders of the item segmentation diagrams and recorded in the order placement database; when a pickup instruction for the target delivery order is received, an order comparison diagram of the grasping position is obtained; from the order placement database, the item segmentation diagram associated with the target delivery order is queried as an order reference diagram for the target delivery order; if the similarity between the order comparison image and the order reference diagram is greater than a preset similarity threshold, the gripper is controlled to grasp the target delivery order from the grasping position to the cargo hold exit of the delivery robot.

[0072] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0073] Perform classification processing based on the item segmentation diagrams to obtain item categories of the item segmentation diagrams;

[0074] Acquire order information of each delivery order of the delivery robot; match the order information of each delivery order with the item category of each item segmentation diagram to obtain a preliminary matching order of each item segmentation diagram; respond to a confirmation operation for the preliminary matching order of the first item segmentation diagram among the item segmentation diagrams, associate the preliminary matching order of the first item segmentation diagram with the first item segmentation diagram and record it in an order placement database.

[0075] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0076] Perform classification processing based on the item segmentation diagrams to obtain item categories of the item segmentation diagrams;

[0077] Acquire order information of each delivery order of the delivery robot; match the item categories of each item segmentation diagram based on the order information of each delivery order to obtain a preliminary pairing order of each item segmentation diagram; respond to a modification operation of the preliminary pairing order of the second item segmentation diagram in each item segmentation diagram, associate the modified pairing order of the second item segmentation diagram with the second item segmentation diagram and record it in the order placement database.

[0078] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0079] The target storage board is photographed to obtain a current image of the target storage board; from the order placement database, an item segmentation map associated with the target delivery order is queried as an order reference map of the target delivery order; the current image is matched with the order reference map to obtain an image position of the target delivery order in the current image; based on the image position and a preset image space conversion relationship, the current placement position of the target delivery order in the target storage board is determined.

[0080] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0081] Based on the post-matching image, the paired orders of each item segmentation diagram are positionally identified to obtain the order placement position of each item segmentation diagram; the order placement position of each item segmentation diagram is associated with the paired orders of each item segmentation diagram and recorded in the order placement database; from the order placement database, the order placement position associated with the target delivery order is queried to serve as the current placement position of the target delivery order.

[0082] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0083] Detect whether the previous delivery order of the target delivery order has been picked up; when it is detected that the previous delivery order has been picked up, control the movement of the target placement plate according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate.

[0084] It should be noted that, those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding process in the following delivery robot hatch control method embodiment, and will not be repeated here.

[0085] In the following, Figure 1 The electronic device shown in is used as an example of the execution subject of the delivery robot hatch control method, and the delivery robot hatch control method provided by the embodiment of the present application is described in detail. For the sake of simplicity and convenience of description, the execution subject will be omitted in the subsequent method embodiments. Figure 1 The scenario is only used to explain the delivery robot door control method provided in the embodiment of the present application, but does not constitute a limitation on the application scenario of the delivery robot door control method provided in the embodiment of the present application.

[0086] See also Figure 2 , Figure 2 201 to 205, wherein:

[0087] 201. Determine a target storage plate where a target delivery order of the delivery robot is located.

[0088] Among them, the delivery order can be an express order, a takeaway order, or goods (for example, in a hotel delivery scenario, the delivery order can be guest room daily necessities), etc.

[0089] Among them, the target delivery order is the order currently prepared for delivery.

[0090] In order to better understand this embodiment, the delivery robot in the embodiment of the present application is first introduced below. Figure 3 As shown, the delivery robot is provided with a cargo compartment and a gripper including a plurality of storage plates, and the storage plates can be moved, for example, along a center point axis of the storage plate (the center point axis is an axis passing through a center point O of the storage plate and perpendicular to the ground, such as Figure 3 As shown, Figure 3 The center point of the middle storage plate 1 is O1, the center point of the storage plate 2 is O2, and the dotted line passing through O1 and O2 represents the center axis) rotates clockwise or counterclockwise so that the target delivery order can be moved to the grabbing position of the gripper for easy removal.

[0091] It should be noted that the specific shapes of the cargo hold and the storage board can be set according to the actual scene requirements. Figure 3 For the sake of convenience of explanation, the cargo hold is shown to be a cuboid and the storage board is a rectangle. In some cases (such as to facilitate the rotation of the storage board and make full use of the cargo hold space), the storage board can be circular and the cargo hold can be cylindrical.

[0092] The target shelf refers to the shelf where the target delivery order is located.

[0093] In order to better understand this embodiment, the following takes the hotel delivery system as an example to illustrate the delivery process of the delivery robot. The hotel delivery system may include a front end, a back end, a delivery robot, and an order end (such as an order applet), as follows:

[0094] 1. Place an order: After the user places an order through the order terminal (such as the order applet), the new order will be sent to the backend.

[0095] 2. Generate order scheduling information: The backend adds the new order as an unscheduled order to the order pool to be scheduled, and generates a pickup code for the unscheduled order.

[0096] 3. Enter the delivery page: The delivery personnel scan the QR code on the delivery robot through the front end (such as mobile phone, bar gun, etc.) to enter the delivery robot's delivery page, and the front end requests the current delivery robot information from the back end. On the delivery page provided by the front end, the delivery personnel can view the delivery robot information (such as the abnormal status of the delivery robot, delivery status, order status, etc.), complete the delivery robot delivery process (such as selecting delivery orders and delivery slots for the delivery robot), etc.

[0097] 4. The backend queries the current delivery robot information (including but not limited to order status, delivery robot status, etc.) and returns it to the frontend.

[0098] 5. After the delivery personnel sees the current delivery robot information through the front end, they choose whether to deliver goods to the current delivery robot according to the situation. When it is necessary to deliver goods to the current delivery robot, the delivery operation process is carried out (including but not limited to selecting delivery orders and delivery slots); at this time, the front end will respond to the delivery personnel's delivery operation and send a delivery request backward.

[0099] 6. Picking: After the backend receives the picking request from the frontend, it sends a picking instruction to the current delivery robot, so that the current delivery robot can start the picking task (including but not limited to opening the hatch, waiting for picking, closing the hatch, broadcasting the door opening and closing voice, etc.).

[0100] 7. Delivery: After the distribution is completed, the distribution staff triggers the delivery operation on the front end (such as clicking the "Delivery" button on the front end); at this time, the front end sends a delivery request to the back end.

[0101] 8. After receiving the delivery request, the backend sends a delivery instruction to the current delivery robot, so that the current delivery robot enters the delivery task.

[0102] 9. The current delivery robot enters the delivery task and returns the delivery result to the backend; the backend returns the delivery result to the frontend.

[0103] 10. After the user triggers the pickup operation through the order end (such as the ordering mini program) (such as the user clicks the "open the door to pick up the goods" button in the ordering mini program, enters the pickup code, etc.), the order end (such as the ordering mini program) sends a pickup request to the backend; after receiving the pickup request, the backend sends a pickup instruction to the current delivery robot, so that the current delivery robot opens the hatch and completes the pickup.

[0104] There are many ways to determine the target storage board in step 201, illustratively including:

[0105] (1) In some embodiments, each shelf can be photographed and identified in real time to determine the target shelf where the target delivery order is located. In this case, step 201 can specifically include the following steps 2011A to 2013A:

[0106] 2011A. Photograph each storage plate of the delivery robot to obtain images of each storage plate.

[0107] 2012A. Perform order recognition on the images of each storage plate to obtain order segmentation maps of the images of each storage plate.

[0108] 2013A. Matching the order segmentation graph based on the image of each shelf with the target delivery order to determine the target shelf where the target delivery order is located.

[0109] (2) In some embodiments, when allocating goods, each delivery order and the shelf where each delivery order is located may be associated and recorded in an order placement database. In step 201, the target shelf where the target delivery order is located may be directly queried from the order placement database. In this case, step 201 may specifically include the following step 211B:

[0110] 2011B. Query the storage board associated with the target delivery order from the order placement database to serve as the target storage board for the target delivery order.

[0111] In some embodiments, the picker can directly select one or more delivery orders and associate them with the target storage board and record them in the order placement database, as shown in Table 1. Similarly, the picker can associate the corresponding delivery orders for each storage board, thereby realizing the association record of each delivery order with the storage board where each delivery order is located and recording it in the order placement database.

[0112] Table 1

[0113] Storage board Delivery Orders Item segmentation diagram Order placement location A 1 P1 A1 A 2 P2 A2 B 3 P3 B3 B 4 P4 B4 …… …… …… ……

[0114] In some embodiments, Figure 4 As shown, the storage board where each delivery order is located can be associated and recorded in the order placement database with reference to the following steps 401 to 403.

[0115] 401. After the target storage board is allocated, an image of the target storage board after the allocation is completed is obtained.

[0116] For example, taking a distribution system including a front end, a back end, a delivery robot, and an order end as an example, the delivery personnel scan the QR code labeled on the delivery robot through the front end (such as a mobile phone, a bar gun, etc.) to enter the delivery robot's delivery page; select the order delivered by the current delivery robot on the delivery robot's delivery page, where the delivery process is as follows: the current delivery robot opens the cargo hold door, the delivery personnel puts each delivery order of the current delivery robot into the cargo hold of the current delivery robot, and the current delivery robot closes the cargo hold door, thereby completing the target shelf delivery. Among them, when delivering, the delivery personnel can put each delivery order into any shelf in the cargo hold. After the delivery is completed, the image of each shelf can be taken to identify the shelf where each delivery order is located, and each delivery order can be associated with the shelf where each delivery order is located and recorded in the order placement database. For example, after the delivery is completed, the image of the target shelf can be taken as the post-delivery image of the target shelf.

[0117] Exemplarily, a camera may be installed above each shelf, and the camera's field of view may cover the entire shelf placement area to ensure that the orders on the shelf can be clearly captured, thereby facilitating identification of the placement location of the delivery order on the shelf.

[0118] The post-allocation image refers to an image captured of the target placement board after the placement of the target placement board is completed.

[0119] 402. Perform segmentation processing on the image after goods distribution to obtain a segmentation map of each item in the image after goods distribution.

[0120] For example, image segmentation algorithms such as threshold segmentation, edge detection (such as using the Canny algorithm to identify the boundaries between items), semantic segmentation (such as the U-Net or Mask R-CNN model in the convolutional neural network, which identifies each object in the image through a trained model and generates a corresponding segmentation area for each object) can be used to segment the image after cargo distribution to obtain a segmentation map of each item in the image after cargo distribution. After the segmentation is completed, the boundary of each item in the image after cargo distribution can be marked to generate the segmented item area; each item area can be extracted separately to form a different item segmentation map.

[0121] 403. In response to the order association instruction of each item segmentation diagram, the paired orders of each item segmentation diagram are associated with the target storage board and recorded in the order placement database.

[0122] Among them, the order placement database is a database used to store information such as the storage board where each delivery order is located, the item segmentation map of each delivery order, and the placement position of each delivery order in the storage board.

[0123] In one embodiment, each item segmentation diagram can be displayed on the front-end distribution page, and the distribution personnel can manually select the matching order of each item segmentation diagram. After the distribution personnel manually selects the matching order of each item segmentation diagram, the order association instruction of each item segmentation diagram will be automatically triggered; at this time, step 403 may specifically include: in response to the order selection operation of the target item segmentation diagram in the item segmentation diagram, triggering the order association instruction of each item segmentation diagram, and associating the selected order of the order selection operation with the target storage board and recording it in the order placement database.

[0124] In one embodiment, in order to improve the matching efficiency between delivery orders and storage boards, the category of items can also be identified based on the segmentation diagrams of each item; based on the order information (such as the order title) and the category of items in the segmentation diagrams of each item, the matching relationship between the delivery order and the storage board is recommended on the front-end distribution page, and the distribution personnel only need to click to confirm.

[0125] 202. Obtain the current placement position of the target delivery order in the target storage board.

[0126] The current placement position refers to the position of the target delivery order in the target storage board.

[0127] There are many ways to implement step 202, which include, for example:

[0128] (1) In some embodiments, the target delivery order may be placed in a different position due to various reasons during the delivery process. In order to improve the accuracy of the current placement of the target delivery order in the target placement board, thereby improving the capture accuracy of the target delivery order, the target placement board may be photographed and identified in real time to determine the current placement of the target delivery order in the target placement board. At this time, step 202 may specifically include steps 2021A to 2024A:

[0129] 2021A. Photograph the target storage board to obtain a current image of the target storage board.

[0130] 2022A. Query the item segmentation map associated with the target delivery order from the order placement database as the order reference map of the target delivery order.

[0131] 2023A. Match the current image with the order reference image to obtain the image position of the target delivery order in the current image.

[0132] Exemplarily, by matching the current image with an order reference image retrieved from a database, the target item can be identified through image matching technology (such as template matching, feature point matching, or convolutional neural network (CNN). Specifically, the items in the order reference image are matched with the items in the current image (such as comparing image similarities), so that the specific location of the items in the target delivery order in the current image can be determined as the image location of the target delivery order in the current image.

[0133] 2024A. Based on the image position and a preset image space conversion relationship, determine the current placement position of the target delivery order in the target storage board.

[0134] (2) In some embodiments, as shown in Table 1, when distributing goods, the order placement position of each item segmentation diagram (for example, order placement position A1 in Table 1) and the paired order of each item segmentation diagram (for example, delivery order 1 in Table 1) can be associated and recorded in the order placement database. In step 202, the current placement position of the target delivery order in the target storage board can be directly queried from the order placement database. At this time, step 202 can specifically include the following step 221B:

[0135] 2021B. Query the order placement location associated with the target delivery order from the order placement database as the current placement location of the target delivery order.

[0136] Exemplarily, referring to the following steps A1 to A2, the order placement position of each item segmentation diagram and the paired order of each item segmentation diagram may be associated and recorded in the order placement database.

[0137] A1. Based on the post-matching image, position recognition is performed on the paired orders of the item segmentation diagrams to obtain order placement positions of the item segmentation diagrams.

[0138] The order placement position refers to the placement position of the matching orders of each item segmentation diagram in the target configuration board.

[0139] For example, first, image processing and feature extraction are performed: by analyzing the images after sorting and the segmentation maps of each item, the location features of each item are extracted; for example, the center position of the item (i.e., the matching order of the item segmentation map) is located by a bounding box, or a deep learning model (such as YOLO, Mask R-CNN, etc.) is used to accurately identify the location of the item. Then, the item placement position is mapped: according to the identified item position and the structure of the storage board, the item is mapped to the specific position in the target configuration board, thereby obtaining the order placement position of each item segmentation map.

[0140] A2. Associating the order placement positions of the item segmentation diagrams with the paired orders of the item segmentation diagrams into the order placement database.

[0141] 203. Control the target storage plate to move according to the current placement position, so that the target delivery order is aligned with the grabbing position of the target storage plate.

[0142] It is understandable that different storage boards have different heights, so each storage board corresponds to a different grabbing position. Figure 3 As shown, the grabbing position of each storage board can be uniformly set on the right side. The gripper can be freely extended to the grabbing position of each storage board to realize grabbing orders from each storage board. Specifically, according to the current placement position, the relative relationship between the position of the target delivery order and the grabbing position is analyzed. If the position of the target delivery order does not meet the requirements of the grabbing position, it is necessary to control the target storage board to move to the correct grabbing position so that the target delivery order is aligned with the grabbing position of the target storage board.

[0143] In some embodiments, in step 203, after receiving the pickup instruction of the target delivery instruction, the target placement plate may be controlled to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate.

[0144] In some embodiments, in order to improve the efficiency of picking up, in step 203, it can be detected whether the previous delivery order of the target delivery order has been picked up; when it is detected that the previous delivery order has been picked up, the target storage plate is controlled to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target storage plate. In this way, after the delivery robot completes the delivery of the current order, it controls the next delivery order to align with the position of the storage plate, so that when it arrives at the delivery address of the next delivery order, it can directly grab the order for the user, improve the efficiency of picking up, and then improve the overall delivery efficiency of the delivery robot.

[0145] 204. After receiving the pickup instruction of the target delivery order, control the gripper to grab the target delivery order to the cargo compartment exit of the delivery robot.

[0146] In some embodiments, upon receiving a pickup instruction for a target delivery order, the gripper is directly controlled to grab the target delivery order to the cargo compartment exit of the delivery robot.

[0147] In some embodiments, in order to avoid mistaken order picking, when receiving the pickup instruction of the target delivery order, the order comparison picture at the gripping position of the gripper will be photographed, and the order comparison picture at the gripping position will be compared with the order reference picture of the target delivery order. Only when the similarity is high will it be considered as the target delivery order, and only then will the gripper be controlled to clamp the delivery order at the gripping position. At this time, step 204 can specifically include the following steps 2041-2043:

[0148] 2041. After receiving the pickup instruction of the target delivery order, obtain the order comparison chart of the grabbing location.

[0149] For example, taking a delivery system including a front-end, a back-end, a delivery robot, and an order end as an example, after a user triggers a pickup operation for a target delivery order through the order end (such as an order applet) (such as a user clicking the "open the door to pick up" button in the order applet, entering a pickup code, etc.), the order end (such as an order applet) sends a pickup request for the target delivery order to the back-end; after receiving the pickup request, the back-end sends a pickup instruction for the target delivery order to the delivery robot. After receiving the pickup instruction for the target delivery order, the delivery robot will call the camera above the target shelf where the target delivery order is located to capture an image of the grabbing position of the target shelf as an order comparison chart of the grabbing position.

[0150] 2042. Query the item segmentation diagram associated with the target delivery order from the order placement database as an order reference diagram for the target delivery order.

[0151] Exemplarily, as shown in Table 1, after the target storage board is allocated, the allocated image of the target storage board is obtained; segmentation processing is performed based on the allocated image to obtain segmentation maps of each item in the allocated image; and the segmentation maps of each item (for example, the item segmentation map P1 in Table 1) are associated with the paired orders of the item segmentation maps (for example, delivery order 1 in Table 1) and recorded in the order placement database.

[0152] There are many ways to “record the item segmentation diagrams and the paired orders of the item segmentation diagrams in the order placement database in association with each other”, exemplarily including the following ways ① and ②:

[0153] ① In some embodiments, in order to improve the matching efficiency between the delivery order and the item segmentation diagram, the item category can also be identified according to each item segmentation diagram; according to the order information (such as the order title) and the item category of each item segmentation diagram, the matching relationship between the delivery order and the item segmentation diagram is recommended on the front-end distribution page, and the distribution personnel only need to click to confirm. At this time, "associating the item segmentation diagrams with the matching orders of the item segmentation diagrams into the order placement database" may include the following steps B1 to B4:

[0154] B1. Perform classification processing based on each item segmentation diagram to obtain the item category of each item segmentation diagram.

[0155] Exemplarily, each item segmentation map can be classified by a pre-trained classification model to obtain the item category of the item segmentation map. For example, a classification model can be constructed using a feature extraction module (such as a convolutional neural network CNN based on deep learning) and a classifier (such as a Softmax layer or a support vector machine SVM), and the feature extraction module (such as a convolutional neural network based on deep learning) is used to extract features from each item segmentation map to obtain the image features of each item segmentation map. Then, the classifier is used to determine the item category in each item segmentation map based on the image features of each item segmentation map. For example, a pre-trained classification model can determine whether the items in each item segmentation map are "milk tea drinks", "meals", "daily necessities", or "others" based on the color, shape, texture and other features of the items.

[0156] B2. Obtain order information of each delivery order of the delivery robot.

[0157] The order information may include but is not limited to: order title, item name, quantity, delivery address, order product ID and other information.

[0158] B3. Matching the order information of each delivery order with the item category of each item segmentation diagram to obtain a preliminary matching order of each item segmentation diagram.

[0159] The preliminary matching order is based on matching the order information of each delivery order with the item category of each item segmentation diagram to obtain the matching order of each item segmentation diagram.

[0160] For example, if the title of a delivery order i contains "milk tea", the item area marked as "milk tea" in all item segmentation maps will be matched with the order information (such as quantity) of the delivery order i. If the item area marked as "milk tea" in an item segmentation map j is matched with the order information (such as quantity) of the delivery order i, then the delivery order i will be used as the preliminary matching order of the item segmentation map j, and so on, thereby obtaining the preliminary matching orders of each item segmentation map.

[0161] B4. In response to a confirmation operation of the preliminary pairing order of the first item segmentation diagram among the item segmentation diagrams, the preliminary pairing order of the first item segmentation diagram is associated with the first item segmentation diagram and recorded in an order placement database.

[0162] ② In some embodiments, in order to improve the matching efficiency between the delivery order and the item segmentation diagram, the item category can also be identified according to each item segmentation diagram; according to the order information (such as the order title) and the item category of each item segmentation diagram, the matching relationship between the delivery order and the item segmentation diagram is recommended on the front-end distribution page, and the distribution personnel can modify the matching relationship between the delivery order and the item segmentation diagram according to the actual situation. At this time, "associating the item segmentation diagram with the matching order of the item segmentation diagram and recording it in the order placement database" may include the following steps C1 to C4:

[0163] C1. Perform classification processing based on each item segmentation diagram to obtain the item category of each item segmentation diagram.

[0164] C2. Obtaining order information of each delivery order of the delivery robot.

[0165] C3. Matching the order information of each delivery order with the item category of each item segmentation diagram to obtain a preliminary matching order of each item segmentation diagram.

[0166] The implementation of steps C1 to C3 is similar to that of steps B1 to B3. For details, please refer to the relevant descriptions in the previous text and will not be repeated here.

[0167] C4. In response to the modification operation of the preliminary pairing order of the second item segmentation diagram among the item segmentation diagrams, the modified pairing order of the second item segmentation diagram is associated with the second item segmentation diagram and recorded in the order placement database.

[0168] 2043. If the similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, control the gripper to grab the target delivery order to the cargo hold exit.

[0169] Exemplarily, the similarity between the order comparison image and the order reference image is compared; if the similarity between the order comparison image and the order reference image is greater than the preset similarity threshold, it proves that the grasping position is the target delivery order, and the gripper is controlled to grasp the target delivery order to the cargo hold exit. If the similarity between the order comparison image and the order reference image is less than or equal to the preset similarity threshold, it proves that the grasping position is not the target delivery order, and the current placement position of the target delivery order in the target storage board can be further identified and updated, and the target storage board is controlled to move according to the updated current placement position, so that the target delivery order is aligned with the grasping position of the target storage board; until the similarity between the order comparison image and the order reference image at the grasping position is greater than the preset similarity threshold, the gripper is controlled to grasp the target delivery order to the cargo hold exit.

[0170] 205. When the target delivery order is at the cargo hold exit, control the cargo hold door of the delivery robot to open.

[0171] When the target delivery order is grabbed to the cargo hold exit, the cargo hold door of the delivery robot is controlled to open, and the user can pick up the goods at this time; after the user finishes picking up the goods, the cargo hold door of the delivery robot is controlled to close.

[0172] From the above content, it can be seen that by controlling the movement of the target placement board where the target delivery order of the delivery robot is located according to the current placement position of the target delivery order in the target placement board, the target delivery order is aligned with the grabbing position of the target placement board; after receiving the pick-up instruction of the target delivery order, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo hold exit of the delivery robot; in this way, when multiple orders are mixed, the target delivery order can be accurately controlled to move to the cargo hold exit, so that multiple orders can be mixed and placed on the same placement board, making full use of the cargo hold storage capacity; when the target delivery order is at the cargo hold exit, the cargo hold door of the delivery robot is controlled to open, so that the picker can pick up the target delivery order to avoid the problem of other orders being picked up by mistake; it can be seen that picking up through this door control method allows orders to be mixed, improves the utilization rate of the robot's cargo hold, and improves the overall delivery efficiency of the robot.

[0173] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned delivery robot door control method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0174] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored, and the computer program can be loaded by a processor to execute any of the delivery robot door control methods provided in the embodiments of the present application. For example, the computer program can be loaded by a processor to execute the following steps:

[0175] Determine a target storage plate where a target delivery order of the delivery robot is located; obtain a current placement position of the target delivery order on the target storage plate; control the movement of the target storage plate according to the current placement position so that the target delivery order is aligned with a grabbing position of the target storage plate; upon receiving a pickup instruction for the target delivery order, control the gripper to grab the target delivery order from the grabbing position to an exit of a cargo hold of the delivery robot; and when the target delivery order is at the exit of the cargo hold, control the cargo hold door of the delivery robot to open.

[0176] In some embodiments, the computer program can be loaded by a processor to perform the following steps:

[0177] After the target storage plate is matched, a matched image of the target storage plate is obtained; segmentation processing is performed on the matched image to obtain segmentation diagrams of each item in the matched image; in response to order association instructions of the segmentation diagrams of each item, the paired orders of the segmentation diagrams of each item are associated with the target storage plate and recorded in an order placement database; from the order placement database, the storage plate associated with the target delivery order is queried to serve as the target storage plate of the target delivery order.

[0178] In some embodiments, the computer program can be loaded by a processor to perform the following steps:

[0179] The item segmentation diagrams are associated with the paired orders of the item segmentation diagrams and recorded in the order placement database; when a pickup instruction for the target delivery order is received, an order comparison diagram of the grasping position is obtained; from the order placement database, the item segmentation diagram associated with the target delivery order is queried as an order reference diagram for the target delivery order; if the similarity between the order comparison image and the order reference diagram is greater than a preset similarity threshold, the gripper is controlled to grasp the target delivery order from the grasping position to the cargo hold exit of the delivery robot.

[0180] In some embodiments, the computer program can be loaded by a processor to perform the following steps:

[0181] Perform classification processing based on the item segmentation diagrams to obtain item categories of the item segmentation diagrams;

[0182] Acquire order information of each delivery order of the delivery robot; match the order information of each delivery order with the item category of each item segmentation diagram to obtain a preliminary matching order of each item segmentation diagram; respond to a confirmation operation for the preliminary matching order of the first item segmentation diagram among the item segmentation diagrams, associate the preliminary matching order of the first item segmentation diagram with the first item segmentation diagram and record it in an order placement database.

[0183] In some embodiments, the computer program can be loaded by a processor to perform the following steps:

[0184] Perform classification processing based on the item segmentation diagrams to obtain item categories of the item segmentation diagrams;

[0185] Acquire order information of each delivery order of the delivery robot; match the item categories of each item segmentation diagram based on the order information of each delivery order to obtain a preliminary pairing order of each item segmentation diagram; respond to a modification operation of the preliminary pairing order of the second item segmentation diagram in each item segmentation diagram, associate the modified pairing order of the second item segmentation diagram with the second item segmentation diagram and record it in the order placement database.

[0186] In some embodiments, the computer program can be loaded by a processor to perform the following steps:

[0187] The target storage board is photographed to obtain a current image of the target storage board; from the order placement database, an item segmentation map associated with the target delivery order is queried as an order reference map of the target delivery order; the current image is matched with the order reference map to obtain an image position of the target delivery order in the current image; based on the image position and a preset image space conversion relationship, the current placement position of the target delivery order in the target storage board is determined.

[0188] In some embodiments, the computer program can be loaded by a processor to perform the following steps: based on the post-matching image, position identification is performed on the paired orders of each item segmentation diagram to obtain the order placement position of each item segmentation diagram; the order placement position of each item segmentation diagram is associated with the paired orders of each item segmentation diagram and recorded in the order placement database; from the order placement database, the order placement position associated with the target delivery order is queried as the current placement position of the target delivery order.

[0189] In some embodiments, the computer program can be loaded by a processor to perform the following steps:

[0190] Detect whether the previous delivery order of the target delivery order has been picked up; when it is detected that the previous delivery order has been picked up, control the movement of the target placement plate according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate.

[0191] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0192] In the above-mentioned delivery robot hatch control method, computer-readable storage medium, and electronic device embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described computer-readable storage medium, electronic device, and corresponding units can refer to the description of the delivery robot hatch control method in the above embodiment, and will not be repeated here.

[0193] The above is a detailed introduction to a delivery robot door control method, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for controlling a hatch of a delivery robot, characterized in that: The delivery robot is provided with a cargo compartment including a plurality of storage plates and a gripper, and the method comprises: Determine the target shelf where the target delivery order of the delivery robot is located; Obtaining the current placement position of the target delivery order in the target storage board; Controlling the target placement plate to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate; Upon receiving a pickup instruction for the target delivery order, controlling the gripper to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot; When the target delivery order is at the cargo hold exit, the cargo hold door of the delivery robot is controlled to open.

2. The method for controlling a hatch of a delivery robot according to claim 1, characterized in that: The method further comprises: After the target storage plate is loaded with goods, an image of the target storage plate after loading with goods is acquired; Perform segmentation processing on the image after goods distribution to obtain a segmentation map of each item in the image after goods distribution; In response to the order association instruction of each item segmentation diagram, the paired orders of each item segmentation diagram are associated with the target storage board and recorded in the order placement database; The step of determining a target storage plate for a target delivery order of the delivery robot includes: The storage board associated with the target delivery order is queried from the order placement database to serve as the target storage board of the target delivery order.

3. The method for controlling a hatch of a delivery robot according to claim 2, characterized in that: The method further comprises: Associating each item segmentation diagram with a paired order of each item segmentation diagram and recording it in the order placement database; After receiving the pickup instruction of the target delivery order, controlling the gripper to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot comprises: After receiving the pickup instruction of the target delivery order, obtaining the order comparison chart of the grabbing location; Querying the item segmentation map recorded in association with the target delivery order from the order placement database as an order reference map of the target delivery order; If the similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot.

4. The method for controlling a hatch of a delivery robot according to claim 3, characterized in that: The associating and recording each item segmentation diagram and the paired order of each item segmentation diagram into the order placement database includes: Perform classification processing based on each item segmentation diagram to obtain an item category of each item segmentation diagram; Obtaining order information of each delivery order of the delivery robot; Based on the order information of each delivery order, the item categories of each item segmentation diagram are matched to obtain a preliminary matching order of each item segmentation diagram; In response to a confirmation operation of a preliminary pairing order for a first item segmentation diagram among the item segmentation diagrams, the preliminary pairing order for the first item segmentation diagram is associated with the first item segmentation diagram and recorded in an order placement database.

5. The method for controlling a hatch of a delivery robot according to claim 3, characterized in that: The associating and recording each item segmentation diagram and the paired order of each item segmentation diagram into the order placement database includes: Perform classification processing based on each item segmentation diagram to obtain an item category of each item segmentation diagram; Obtaining order information of each delivery order of the delivery robot; Based on the order information of each delivery order, the item categories of each item segmentation diagram are matched to obtain a preliminary matching order of each item segmentation diagram; In response to a modification operation of the preliminary pairing order of the second item segmentation diagram among the item segmentation diagrams, the modified pairing order of the second item segmentation diagram is associated with the second item segmentation diagram and recorded in the order placement database.

6. The method for controlling a hatch of a delivery robot according to claim 3, characterized in that: The obtaining of the current placement position of the target delivery order in the target storage board includes: photographing the target storage board to obtain a current image of the target storage board; Querying the item segmentation map recorded in association with the target delivery order from the order placement database as an order reference map of the target delivery order; Matching the current image with the order reference image to obtain an image position of the target delivery order in the current image; Based on the image position and a preset image space conversion relationship, a current placement position of the target delivery order in the target storage plate is determined.

7. The method for controlling a hatch of a delivery robot according to claim 2, characterized in that: The method further comprises: Based on the post-arrangement image, position recognition is performed on the paired orders of the item segmentation diagrams to obtain order placement positions of the item segmentation diagrams; Associating the order placement positions of the item segmentation diagrams with the paired orders of the item segmentation diagrams into the order placement database; The obtaining of the current placement position of the target delivery order in the target storage board includes: The order placement location recorded in association with the target delivery order is queried from the order placement database to serve as the current placement location of the target delivery order.

8. The method for controlling a hatch of a delivery robot according to claim 1, characterized in that: The step of controlling the target placement plate to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate includes: Detecting whether the previous delivery order of the target delivery order has been picked up; When it is detected that the pickup of the previous delivery order is completed, the target placement plate is controlled to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement plate.

9. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes the delivery robot hatch control method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the delivery robot hatch control method as described in any one of claims 1 to 8.

Images