Delivery robot cabin door control method, delivery robot and readable storage medium

Abstract

The application provides a delivery robot cabin door control method, an electronic device and a computer readable storage medium. The delivery robot cabin door control method comprises the following steps: determining a target storage plate where a target distribution order of a delivery robot is located, the delivery robot being provided with a cargo cabin comprising a plurality of storage plates and a gripper; acquiring a current placement position of the target distribution order in the target storage plate; controlling the target storage plate to move according to the current placement position, so that the target distribution order is aligned with a grabbing position of the target storage plate; when a cargo taking instruction of the target distribution order is received, controlling the gripper to grab the target distribution order from the grabbing position to a cargo cabin outlet of the delivery robot; and when the target distribution order is at the cargo cabin outlet, controlling a cargo cabin door of the delivery robot to open. The application can improve the utilization rate of the robot cargo cabin and improve the overall distribution efficiency of the robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a delivery robot hatch control method, a delivery robot and a computer readable storage medium. BACKGROUND

[0002] With the further maturity of robot technology and the increasing demand of consumers for automated services, robots are applied to more and more scenarios, such as hotels, retail, warehouse logistics, etc. Delivering goods is an important application scenario of robots, for example, hotels usually use robots to deliver daily necessities, food, takeout, etc. to guest rooms.

[0003] In the related art, in order to improve the safety of delivery, the robot is provided with multiple compartments for placing goods, each compartment is used to place an order, and the opening and closing of the hatch of each compartment is controlled separately. However, the present inventors have found in the actual research and development process that some orders are small and only need a small part of the position of the compartment, so most of the capacity positions are empty, resulting in low overall delivery efficiency of the robot. SUMMARY

[0004] The present application provides a delivery robot hatch control method, a delivery robot and a computer readable storage medium, which can improve the utilization rate of the robot cargo hold and improve the overall delivery efficiency of the robot.

[0005] In a first aspect, the present application provides a delivery robot hatch control method, the delivery robot is provided with a cargo hold including multiple storage plates and a gripper, and the method comprises:

[0006] determining a target storage plate where a target delivery order of the delivery robot is located;

[0007] obtaining a current placement position of the target delivery order in the target storage plate;

[0008] controlling the target storage plate to move according to the current placement position, so that the target delivery order is aligned with a grabbing position of the target storage plate;

[0009] after receiving a goods taking instruction of the target delivery order, controlling the gripper to grab the target delivery order from the grabbing position to a cargo hold exit of the delivery robot;

[0010] when the target delivery order is at the cargo hold exit, controlling a cargo hold hatch of the delivery robot to open.

[0011] In some embodiments, the method further comprises:

[0012] after the target storage plate is completed, obtaining an image of the target storage plate after the goods are delivered;

[0013] performing segmentation processing on the post-packing image to obtain each article segmentation graph in the post-packing image;

[0014] in response to order association instructions of the each article segmentation graph, recording the paired order of the each article segmentation graph and the target placement board to an order placement database;

[0015] The method further comprises:

[0016] querying, from the order placement database, a placement board associated with the target delivery order as the target placement board of the target delivery order.

[0017] In some embodiments, the method further comprises:

[0018] recording the each article segmentation graph and the paired order of the each article segmentation graph to the order placement database;

[0019] The method further comprises:

[0020] When the order picking instruction of the target delivery order is received, an order comparison image of the picking position is obtained.

[0021] querying, from the order placement database, an article segmentation graph associated with the target delivery order as an order reference image of the target delivery order;

[0022] If the similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, the gripper is controlled to pick the target delivery order from the picking position to the cargo compartment outlet of the delivery robot.

[0023] In some embodiments, the recording the each article segmentation graph and the paired order of the each article segmentation graph to the order placement database comprises:

[0024] performing classification processing based on the each article segmentation graph to obtain an article category of the each article segmentation graph;

[0025] obtaining order information of each delivery order of the delivery robot;

[0026] performing matching based on the order information of the each delivery order and the article category of the each article segmentation graph to obtain a preliminary paired order of the each article segmentation graph;

[0027] In response to a confirmation operation of the preliminary matching order of the first item segmentation graph, the preliminary matching order of the first item segmentation graph is recorded in the order placement database in association with the first item segmentation graph.

[0028] In some embodiments, the recording of the item segmentation graphs and the matching orders of the item segmentation graphs in the order placement database comprises:

[0029] Based on the item segmentation graphs, a classification processing is performed to obtain item categories of the item segmentation graphs.

[0030] Obtaining order information of each delivery order of the delivery robot;

[0031] Based on the order information of each delivery order and the item categories of the item segmentation graphs, a matching is performed to obtain preliminary matching orders of the item segmentation graphs.

[0032] In response to a modification operation of the preliminary matching order of a second item segmentation graph in the item segmentation graphs, a modified matching order of the second item segmentation graph is recorded in the order placement database in association with the second item segmentation graph.

[0033] In some embodiments, the obtaining of the current placement position of the target delivery order in the target placement board comprises:

[0034] Taking a photograph of the target placement board to obtain a current image of the target placement board;

[0035] From the order placement database, an item segmentation graph recorded in association with the target delivery order is queried as an order reference graph of the target delivery order;

[0036] Matching the current image and the order reference graph to obtain an image position of the target delivery order in the current image;

[0037] Based on the image position and a preset image space conversion relationship, a current placement position of the target delivery order in the target placement board is determined.

[0038] In some embodiments, the method further comprises:

[0039] Based on the after-delivery image, a position recognition is performed on the matching orders of the item segmentation graphs to obtain order placement positions of the item segmentation graphs;

[0040] The order placement positions of the item segmentation graphs and the matching orders of the item segmentation graphs are recorded in the order placement database in association with each other;

[0041] The obtaining the current placement position of the target delivery order in the target tray includes:

[0042] The order placement position associated with the target delivery order is queried from the order placement database as the current placement position of the target delivery order.

[0043] In some embodiments, the controlling the target tray to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target tray includes:

[0044] Detecting whether the previous delivery order of the target delivery order is completed;

[0045] When it is detected that the previous delivery order is completed, the target tray is controlled to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target tray.

[0046] In a second aspect, the present application also provides a delivery robot, which includes a processor and a memory, the memory stores a computer program, and the processor invokes the computer program in the memory to execute any of the delivery robot hatch control methods provided by the present application.

[0047] In a third aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is loaded by a processor to execute the delivery robot hatch control method.

[0048] In the present application, the target tray in which the target delivery order is located is controlled to move according to the current placement position of the target delivery order in the target tray, so that the target delivery order is aligned with the grabbing position of the target tray; when the pickup instruction of the target delivery order is received, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo compartment outlet of the delivery robot; in this way, the target delivery order can be accurately controlled to move to the cargo compartment outlet when multiple orders are mixed, so that multiple orders can be mixed and placed on the same tray, and the cargo compartment storage capacity is fully utilized; when the target delivery order is at the cargo compartment outlet, the cargo compartment hatch of the delivery robot is controlled to open, so that the target delivery order can be picked up by the pickup person, and the problem of mis-picking other orders is avoided; it can be seen that the order can be mixed and placed by this hatch control method for picking up, the utilization rate of the cargo compartment of the robot is improved, and the overall delivery efficiency of the robot is improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced. Obviously, the drawings in the following description only constitute some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0050] Figure 1 is a structural schematic block diagram of a delivery robot provided by an embodiment of the present application;

[0051] Figure 2 is a flow schematic diagram of a delivery robot cabin door control method provided by an embodiment of the present application;

[0052] Figure 3 is a structural framework schematic diagram of a delivery robot in an embodiment of the present application;

[0053] Figure 4 is a flow schematic diagram of recording an order placed on a storage plate provided by an embodiment of the present application;

[0054] Figure 5 is an embodiment schematic diagram of step 204 provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of protection of the present application.

[0056] The flowcharts shown in the drawings are only exemplary descriptions, and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the described order. For example, some operations / steps can be decomposed, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0057] In the description of the embodiments of the present application, it should be understood that the terms “first”, “second” are only used for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first”, “second” can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of “multiple” is two or more, unless otherwise specifically limited.

[0058] The following description is presented to enable any person skilled in the art to practice the application as claimed. Details are set forth in the following description for purpose of explanation. It should be appreciated that one of ordinary skill in the art will realize that the application can be practiced without the use of these specific details. In other instances, well-known processes have not been described in detail in order not to unnecessarily obscure the description of the embodiments of the application. Accordingly, the present application is not intended to be limited by the embodiments shown, but is to be accorded with the widest scope consistent with the principles and features disclosed herein.

[0059] Embodiments of the present application provide a delivery robot cabin door control method, a delivery robot and a computer readable storage medium. The delivery robot can be a robot for delivering food, express delivery, etc.

[0060] Some embodiments of the present application will be described in detail below with reference to the drawings. The following embodiments and features of the embodiments can be combined with each other without conflict.

[0061] Figure 1 is a structural schematic block diagram of an electronic device provided by embodiments of the present application.

[0062] As shown in Figure 1 , the electronic device 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected through a bus 103, such as a PCIe (Peripheral Component Interconnect Express) bus.

[0063] Specifically, the processor 101 is configured to provide computing and control capabilities to support the operation of the entire electronic device 100. The processor 101 can be a central processing unit (CPU), and the processor 101 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0064] Specifically, the memory 102 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0065] Those skilled in the art can understand that Figure 1 The structure shown in the figure is only a block diagram of part of the structure related to the embodiment of the present application, and does not constitute a limitation on the electronic device to which the embodiment of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0066] The processor 101 is configured to run the computer program stored in the memory 102, and implement any one of the delivery robot hatch control methods provided by the embodiments of the present application when the computer program is executed. For example, the processor 101 is configured to run the computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0067] Determine the target storage board where the target delivery order of the delivery robot is located; obtain the current placement position of the target delivery order in the target storage board; control the target storage board to move according to the current placement position, so that the target delivery order is aligned with the grabbing position of the target storage board; when receiving the order picking instruction of the target delivery order, control the gripper to grab the target delivery order from the grabbing position to the cargo compartment outlet of the delivery robot; when the target delivery order is at the cargo compartment outlet, control the cargo compartment hatch of the delivery robot to open.

[0068] In some embodiments, the processor 101 is configured to run the computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0069] After the target storage board is completed, obtain the image of the target storage board after the delivery; according to the image after the delivery, the segmentation processing is carried out, and the segmentation image of each article in the image after the delivery is obtained; in response to the order association instruction of each article segmentation image, the matching order of each article segmentation image is recorded in the order placement database; from the order placement database, query the storage board associated with the target delivery order as the target storage board of the target delivery order.

[0070] In some embodiments, the processor 101 is configured to run the computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0071] The order placement database is used for recording the association between each of the item segmentation images and the matched orders; when the pickup instruction of the target delivery order is received, an order comparison image of the grabbing position is obtained; the item segmentation image recorded in association with the target delivery order in the order placement database is queried as an order reference image of the target delivery order; if the similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo compartment outlet of the delivery robot.

[0072] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0073] The classification processing is performed based on the item segmentation images, and item categories of the item segmentation images are obtained.

[0074] The order information of each delivery order of the delivery robot is obtained; the order information of each delivery order is matched with the item categories of the item segmentation images, and preliminary matched orders of the item segmentation images are obtained; in response to a confirmation operation on the preliminary matched order of a first item segmentation image among the item segmentation images, the preliminary matched order of the first item segmentation image is recorded in association with the first item segmentation image in an order placement database.

[0075] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0076] The classification processing is performed based on the item segmentation images, and item categories of the item segmentation images are obtained.

[0077] The order information of each delivery order of the delivery robot is obtained; the order information of each delivery order is matched with the item categories of the item segmentation images, and preliminary matched orders of the item segmentation images are obtained; in response to a confirmation operation on the preliminary matched order of a first item segmentation image among the item segmentation images, the preliminary matched order of the first item segmentation image is recorded in association with the first item segmentation image in an order placement database.

[0078] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0079] An image of the target placement plate is captured to obtain a current image of the target placement plate; an article segmentation image associated with a record of the target delivery order is queried from the order placement database as an order reference image of the target delivery order; the current image and the order reference image are matched to obtain an image position of the target delivery order in the current image; and a current placement position of the target delivery order in the target placement plate is determined based on the image position and a preset image space conversion relationship.

[0080] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0081] The order placement position of each article segmentation image is recorded in the order placement database; and the order placement position of the target delivery order is queried from the order placement database as a current placement position of the target delivery order.

[0082] In some embodiments, the processor 101 is configured to run a computer program stored in the memory 102, and when the computer program is executed, the following steps can be implemented:

[0083] It is detected whether a previous delivery order of the target delivery order is completed; and when it is detected that the previous delivery order is completed, the target placement plate is controlled to move according to the current placement position, so that the target delivery order is aligned with a grabbing position of the target placement plate.

[0084] It should be noted that, for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding process in the following delivery robot hatch control method embodiments, which will not be described here.

[0085] Hereinafter, the electronic device shown in Figure 1 will be taken as an example of the execution subject of the delivery robot hatch control method to introduce the delivery robot hatch control method provided by the embodiments of the present application in detail. In order to simplify and facilitate the description, the execution subject will be omitted in the following method embodiments. It should be noted that Figure 1 the scenario in only serves to explain the delivery robot hatch control method provided by the embodiments of the present application, but does not constitute a limitation on the application scenarios of the delivery robot hatch control method provided by the embodiments of the present application.

[0086] Please refer to Figure 2 ,Figure 2 is a flowchart of a delivery robot hatch control method provided by an embodiment of the present application. The delivery robot hatch control method comprises steps 201-205, wherein:

[0087] 201, determining a target storage board on which a target delivery order of a delivery robot is located.

[0088] The delivery order can be a courier order, a take-out order, goods (such as a delivery order in a hotel delivery scenario, which can be a room daily supply), etc.

[0089] The target delivery order is an order that is currently ready for delivery.

[0090] To better understand the present embodiment, the delivery robot in the present embodiment will be introduced first as follows. Figure 3 As shown in the figure, the delivery robot is provided with a cargo hold comprising a plurality of storage boards and a gripper, and the storage boards can move, for example, rotate clockwise or counterclockwise about the center point axis of the storage board (the center point axis is an axis passing through the center point O of the storage board and perpendicular to the ground, as shown in Figure 3 Figure 3 The center point of the storage board 1 is O1, the center point of the storage board 2 is O2, and the dashed line passing through O1 and O2 represents the center axis) to move the target delivery order to the gripping position of the gripper for easy removal.

[0091] It should be noted that the specific shape of the cargo hold and the storage board can be set according to actual scene requirements, Figure 3 for example, the cargo hold is a cuboid and the storage board is a rectangle, in some cases (such as to facilitate rotation of the storage board and make full use of the cargo hold space) the storage board can be circular and the cargo hold can be cylindrical.

[0092] The target storage board refers to the storage board on which the target delivery order is located.

[0093] To better understand the present embodiment, the delivery process of the delivery robot will be described below taking a hotel delivery system as an example, which can comprise a front end, a back end, a delivery robot, and an order end (such as a small program for placing an order), as follows:

[0094] 1. Order placement: after a user places an order through the order end (such as a small program for placing an order), the new order will be sent to the back end.

[0095] 2. Order scheduling information generation: the back end adds the new order as an unscheduled order to the pool of orders to be scheduled and generates a pick-up code for the unscheduled order.

[0096] ​3. Entering the delivery page: the delivery personnel scans the two-dimensional code on the delivery robot through the front end (such as a mobile phone, a gun, etc.), enters the delivery page of the delivery robot, and the front end requests the current delivery robot information to the back end. On the delivery page provided by the front end, the delivery personnel can view the delivery robot information (such as the abnormal state of the delivery robot, the delivery state, the order state, etc.), complete the delivery process of the delivery robot (such as selecting a delivery order and a delivery compartment for the delivery robot), etc.

[0097] 4. The back end queries the current delivery robot information (including but not limited to order state, delivery robot state, etc.) and returns it to the front end.

[0098] 5. After the delivery personnel sees the current delivery robot information through the front end, they can choose whether to deliver the current delivery robot according to the situation. When the current delivery robot needs to be delivered, the delivery operation process (including but not limited to selecting a delivery order and a delivery compartment) is performed. At this time, the front end will respond to the delivery operation of the delivery personnel and send a delivery request to the back end.

[0099] 6. Delivery: After the back end receives the delivery request from the front end, it sends a delivery instruction to the current delivery robot; so that the current delivery robot enters the delivery task (including but not limited to opening the door, waiting for delivery, closing the door, and broadcasting the opening and closing door voice, etc.).

[0100] 7. Delivery: After the delivery is completed, the delivery personnel triggers the delivery operation on the front end (such as clicking the "delivery" button on the front end). At this time, the front end sends a delivery request to the back end.

[0101] 8. After the back end receives the delivery request, it sends a delivery instruction to the current delivery robot; so that the current delivery robot enters the delivery task.

[0102] 9. The current delivery robot enters the delivery task and returns the delivery result to the back end; the back end returns the delivery result to the front end.

[0103] 10. After the user triggers the pickup operation (such as the user clicking the "open door to pick up" button on the order applet and inputting the pickup code) through the order end (such as the order applet), the order end (such as the order applet) sends a pickup request to the back end; after the back end receives the pickup request, it sends a pickup instruction to the current delivery robot, so that the current delivery robot opens the door and completes the pickup.

[0104] There are various ways to determine the target storage plate in step 201, and exemplarily, they include:

[0105] (1) In some embodiments, the target delivery order plate can be determined by real-time photographing and identification of each storage plate. At this time, step 201 can specifically include steps 2011A-2013A as follows:

[0106] 2011A. Take pictures of each shelf of the delivery robot to obtain images of each shelf.

[0107] 2012A. Perform order recognition on the images of each shelf to obtain an order segmentation map of each shelf image.

[0108] 2013A. Match the order segmentation map based on the images of each shelf with the target delivery order to determine the target shelf where the target delivery order is located.

[0109] (2) In some embodiments, during order fulfillment, each delivery order can be associated with and recorded in the order placement database along with the shelf where each delivery order is located. In step 201, the target shelf where the target delivery order is located can be directly queried from the order placement database. In this case, step 201 may specifically include the following step 2011B:

[0110] 2011B. From the order placement database, query the storage board associated with the target delivery order, and use it as the target storage board for the target delivery order.

[0111] In some embodiments, the picking personnel can directly select one or more delivery orders and associate them with the target shelf to record in the order placement database, as shown in Table 1. Similarly, the picking personnel can associate the corresponding delivery orders with each shelf, thus realizing the association of each delivery order with the shelf where each delivery order is located to the order placement database.

[0112] Table 1

[0113] Shelf Delivery order Item split map Order placement location A 1 P1 A1 A 2 P2 A2 B 3 P3 B3 B 4 P4 B4 …… …… …… ……

[0114] In some embodiments, such as Figure 4 As shown, you can refer to steps 401 to 403 below to associate and record the shelf where each delivery order is located to the order placement database.

[0115] 401. After the target shelf is loaded with goods, obtain an image of the target shelf after loading.

[0116] For example, taking a delivery system including a front end, a back end, a delivery robot and an order end as an example, the delivery personnel scans the two-dimensional code marked on the delivery robot through the front end (such as a mobile phone, a gun, etc.), enters the delivery page of the delivery robot, selects the order to be delivered by the current delivery robot on the delivery page of the current delivery robot, and the delivery process is as follows: the current delivery robot opens the cargo compartment door, the delivery personnel puts the delivery orders of the current delivery robot into the cargo compartment of the current delivery robot, and the current delivery robot closes the cargo compartment door, thereby completing the delivery of the target placement board. Wherein, the delivery personnel can put each delivery order into any placement board of the cargo compartment during delivery, and after the delivery is completed, the image of each placement board can be shot for identifying the placement board where each delivery order is placed, and each delivery order and the placement board where each delivery order is placed are recorded in the order placement database, for example, the image of the target placement board can be shot as the post-delivery image of the target placement board after the delivery is completed.

[0117] Exemplarily, a camera can be installed above each placement board, and the field of view of the camera covers the entire placement area of the placement board, so as to ensure that the order on the placement board can be clearly shot, and the placement position of the delivery order in the placement board can be identified.

[0118] Wherein, the post-delivery image refers to the image of the target placement board shot after the delivery of the target placement board is completed.

[0119] 402. Perform segmentation processing on the post-delivery image to obtain each item segmentation graph in the post-delivery image.

[0120] Exemplarily, image segmentation algorithms such as threshold segmentation, edge detection (such as using Canny algorithm to identify the boundary between items), semantic segmentation (such as U-Net or Mask R-CNN model in convolutional neural network, identifying each object in the image through the trained model, and generating the corresponding segmentation area for each object) can be used to perform segmentation processing on the post-delivery image to obtain each item segmentation graph in the post-delivery image. After segmentation, the boundary of each item in the post-delivery image can be marked to generate the segmented item area; each item area can be extracted separately to form different item segmentation graphs.

[0121] 403. In response to the order association instruction of each item segmentation graph, the matched order of each item segmentation graph is recorded in the order placement database.

[0122] Wherein, the order placement database is a database for storing information such as the placement board where each delivery order is placed, the item segmentation graph of each delivery order, the placement position of each delivery order in the placement board, etc.

[0123] In an embodiment, each item segmentation image can be displayed on a delivery page of a front end, a delivery personnel can manually select a matched order for each item segmentation image, and after the delivery personnel manually selects the matched order for each item segmentation image, an order association instruction for each item segmentation image is automatically triggered; at this time, step 403 can specifically include: in response to an order selection operation of a target item segmentation image in the each item segmentation image, triggering the order association instruction for each item segmentation image, and recording the selected order of the order selection operation and the target placement board in the order placement database.

[0124] In an embodiment, in order to improve the matching efficiency between the delivery order and the placement board, the item category can also be identified according to each item segmentation image; according to the order information (such as the order title) and the item category of each item segmentation image, the matching relationship between the delivery order and the placement board is recommended on the delivery page of the front end, and the delivery personnel only needs to click to confirm.

[0125] 202, obtaining the current placement position of the target delivery order in the target placement board.

[0126] The current placement position refers to the position of the target delivery order in the target placement board.

[0127] There are various implementation manners of step 202, and exemplarily, including:

[0128] (1) In some embodiments, the placement position of the target delivery order can be changed due to various reasons during the delivery process, in order to improve the accuracy of the current placement position of the target delivery order in the target placement board, and thus improve the grabbing accuracy of the target delivery order, the target placement board can be photographed and recognized in real time to determine the current placement position of the target delivery order in the target placement board. At this time, step 202 can specifically include 2021A-2024A:

[0129] 2021A, photographing the target placement board to obtain a current image of the target placement board.

[0130] 2022A, querying an item segmentation image associated with the target delivery order from the order placement database as an order reference image of the target delivery order.

[0131] 2023A, matching the current image and the order reference image to obtain an image position of the target delivery order in the current image.

[0132] Exemplarily, the current image is matched with the order reference image queried from the database, and the target item can be identified through image matching technology (such as template matching, feature point matching or convolutional neural network CNN), and specifically, the item in the order reference image is matched with the item in the current image (such as comparing image similarity), so that the specific position of the item in the target delivery order in the current image can be determined as the image position of the target delivery order in the current image.

[0133] 2024A, based on the image position and the preset image space conversion relationship, determining the current placement position of the target delivery order in the target placement board.

[0134] (2) In some embodiments, as shown in Table 1, the order placement position of each item segmentation image (such as order placement position A1 in Table 1) can be associated with the matched order of each item segmentation image (such as delivery order 1 in Table 1) and recorded to the order placement database during the distribution, and the current placement position of the target delivery order in the target placement board can be directly queried from the order placement database in step 202. At this time, step 202 can specifically include the following step 2021B:

[0135] 2021B, querying the order placement position associated with the target delivery order from the order placement database as the current placement position of the target delivery order.

[0136] Exemplarily, the order placement position of each item segmentation image and the matched order of each item segmentation image can be associated and recorded to the order placement database in the following steps A1-A2 manner.

[0137] A1, based on the post-distribution image, performing position identification on the matched order of each item segmentation image to obtain the order placement position of each item segmentation image.

[0138] Wherein, the order placement position refers to the placement position of the matched order of each item segmentation image in the target configuration board.

[0139] Exemplarily, first, image processing and feature extraction are performed: by analyzing the post-distribution image and each item segmentation image, the position feature of each item is extracted; for example, the center position of the item (i.e. the matched order of the item segmentation image) is located by the bounding box, or the position of the item is accurately identified using a deep learning model (such as YOLO, Mask R-CNN, etc.). Then, the item placement position mapping is performed: according to the identified item position and the structure of the placement board, the item is mapped to the specific position in the target configuration board, so as to obtain the order placement position of each item segmentation image.

[0140] A2, record the association between the order placement position of each item segmentation graph and the matched order of each item segmentation graph in the order placement database.

[0141] 203, control the target placement board to move according to the current placement position, so that the target delivery order is aligned with the grabbing position of the target placement board.

[0142] It can be understood that different placement boards have different heights, so the grabbing position corresponding to each placement board is different. For example, as shown in FIG. 8, the grabbing position of each placement board can be uniformly set on the right side. The gripper can freely extend to the grabbing position of each placement board to realize grabbing the order from each placement board. Specifically, according to the current placement position, the relative relationship between the position of the target delivery order and the grabbing position is analyzed. If the position of the target delivery order does not meet the requirements of the grabbing position, the target placement board needs to be controlled to move to the correct grabbing position, so that the target delivery order is aligned with the grabbing position of the target placement board. Figure 3

[0143] In some embodiments, in step 203, after receiving the pickup instruction of the target delivery instruction, the target placement board can be controlled to move according to the current placement position, so that the target delivery order is aligned with the grabbing position of the target placement board.

[0144] In some embodiments, in order to improve the pickup efficiency, in step 203, it can be detected whether the last delivery order of the target delivery order is completed. When it is detected that the last delivery order is completed, the target placement board can be controlled to move according to the current placement position, so that the target delivery order is aligned with the grabbing position of the target placement board. In this way, after the delivery robot completes the delivery of the current order, the next delivery order is controlled to be aligned with the position of the placement board, so that when it reaches the pickup address of the next delivery order, it can directly grab the order for the user, improve the pickup efficiency, and further improve the overall delivery efficiency of the delivery robot.

[0145] 204, when receiving the pickup instruction of the target delivery order, control the gripper to grab the target delivery order to the cargo compartment outlet of the delivery robot.

[0146] In some embodiments, when receiving the pickup instruction of the target delivery order, the gripper can be directly controlled to grab the target delivery order to the cargo compartment outlet of the delivery robot.

[0147] ​In some embodiments, in order to avoid order mis-taking, when receiving the pickup instruction of the target delivery order, the order comparison image at the pickup position of the gripper is also shot, and the order comparison image at the pickup position of the gripper is compared with the order reference image of the target delivery order. If the similarity is high, it is considered to be the target delivery order, and then the gripper is controlled to clamp the delivery order at the pickup position. At this time, step 204 can specifically include the following steps 2041-2043:

[0148] 2041. When receiving the pickup instruction of the target delivery order, the order comparison image at the pickup position is obtained.

[0149] Exemplarily, taking a delivery system including a front end, a back end, a delivery robot, and an order end as an example, after a user triggers a pickup operation (such as the user clicking a “door opening and pickup” button or inputting a pickup code in the order placing applet) for a target delivery order through the order end (such as the order placing applet), the order end (such as the order placing applet) sends a pickup request of the target delivery order to the back end. After receiving the pickup request, the back end sends a pickup instruction of the target delivery order to the delivery robot. After receiving the pickup instruction of the target delivery order, the delivery robot will call the camera above the target storage board where the target delivery order is located to shoot the image of the pickup position of the target storage board as the order comparison image of the pickup position.

[0150] 2042. From the order placement database, the item segmentation image associated with the record of the target delivery order is queried as the order reference image of the target delivery order.

[0151] Exemplarily, as shown in Table 1, after the target storage board is completed, the image after the target storage board is completed is obtained; the segmentation processing is performed according to the image after the target storage board is completed to obtain each item segmentation image in the image after the target storage board is completed; and the each item segmentation image (such as the item segmentation image P1 in Table 1) is associated with the matched order (such as the delivery order 1 in Table 1) of the each item segmentation image and recorded in the order placement database.

[0152] There are various ways to “record the each item segmentation image and the matched order of the each item segmentation image in the order placement database”, exemplarily including the following ways ① and ②:

[0153] ① In some embodiments, in order to improve the matching efficiency between the delivery order and the item segmentation image, the item category can also be identified according to each item segmentation image; according to the order information (such as the order title) and the item category of each item segmentation image, the matching relationship between the delivery order and the item segmentation image is recommended on the order page of the front end, and the orderer only needs to click to confirm. At this time, “recording the each item segmentation image and the matched order of the each item segmentation image in the order placement database” can include the following steps B1-B4:

[0154] B1, performing classification processing based on the each item segmentation map to obtain an item category of the each item segmentation map.

[0155] Exemplarily, the item category of the each item segmentation map can be obtained by performing classification processing on the each item segmentation map through a pre-trained classification model. For example, the classification model can be constructed by using a feature extraction module (such as a convolutional neural network CNN based on deep learning) and a classifier (such as a Softmax layer or a support vector machine SVM), the image features of the each item segmentation map are extracted through the feature extraction module (such as a convolutional neural network based on deep learning) respectively, and then the item category in the each item segmentation map is determined by the classifier according to the image features of the each item segmentation map. For example, the pre-trained classification model can determine whether the item in the each item segmentation map is a "milk tea beverage", a "meal", a "daily necessity", or "other" according to the color, shape, texture, and other features of the item.

[0156] B2, obtaining order information of each delivery order of the delivery robot.

[0157] The order information can include but is not limited to order title, item name, quantity, delivery address, order item ID, and other information.

[0158] B3, matching the order information of each delivery order with the item category of the each item segmentation map to obtain a preliminary paired order of the each item segmentation map.

[0159] The preliminary paired order is the paired order of the each item segmentation map obtained by matching the order information of each delivery order with the item category of the each item segmentation map.

[0160] For example, if the title of a certain delivery order i contains "milk tea", the item region marked as "milk tea" in all item segmentation maps will be matched with the order information (such as quantity) of the delivery order i. If the item region marked as "milk tea" in a certain item segmentation map j matches the order information (such as quantity) of the delivery order i, the delivery order i will be taken as the preliminary paired order of the item segmentation map j. Similarly, the preliminary paired orders of the each item segmentation map are obtained.

[0161] B4, in response to a confirmation operation on the preliminary paired order of a first item segmentation map in the each item segmentation map, associating the preliminary paired order of the first item segmentation map with the first item segmentation map to an order placement database.

[0162] In some embodiments, in order to improve the matching efficiency between the delivery order and the item segmentation graph, the item category of each item segmentation graph can also be identified; based on the order information (such as the order title) and the item category of each item segmentation graph, the matching relationship between the delivery order and the item segmentation graph is recommended on the front-end delivery page, and the delivery personnel can modify the matching relationship between the delivery order and the item segmentation graph according to the actual situation. At this time, "associating the each item segmentation graph with the pairing order record of the each item segmentation graph in the order placement database" can include the following steps C1-C4:

[0163] C1, classifying based on the each item segmentation graph to obtain the item category of the each item segmentation graph.

[0164] C2, obtaining the order information of each delivery order of the delivery robot.

[0165] C3, matching based on the order information of the each delivery order and the item category of the each item segmentation graph to obtain the preliminary pairing order of the each item segmentation graph.

[0166] The implementation of steps C1-C3 is similar to that of steps B1-B3, and specific reference can be made to the related description in the foregoing, which will not be repeated here.

[0167] C4, in response to the modification operation of the preliminary pairing order of the second item segmentation graph in the each item segmentation graph, associating the modified pairing order of the second item segmentation graph with the second item segmentation graph in the order placement database.

[0168] 2043, if the similarity between the order comparison image and the order reference image is greater than the preset similarity threshold, the gripper is controlled to grasp the target delivery order to the cargo compartment outlet.

[0169] Exemplarily, the similarity between the order comparison image and the order reference image is compared; if the similarity between the order comparison image and the order reference image is greater than the preset similarity threshold, it is proved that the target delivery order is at the grasping position, and then the gripper is controlled to grasp the target delivery order to the cargo compartment outlet. If the similarity between the order comparison image and the order reference image is less than or equal to the preset similarity threshold, it is proved that the target delivery order is not at the grasping position, and then the current placement position of the target delivery order in the target placement board can be further identified and updated, and the target placement board is controlled to move according to the updated current placement position, so that the target delivery order is aligned with the grasping position of the target placement board; until the similarity between the order comparison image of the grasping position and the order reference image is greater than the preset similarity threshold, the gripper is controlled to grasp the target delivery order to the cargo compartment outlet.

[0170] 205, when the target delivery order is at the cargo compartment outlet, the cargo compartment door of the delivery robot is controlled to open.

[0171] When the target delivery order is grabbed to the cargo compartment outlet, the cargo compartment door of the delivery robot is opened, at which time the user can take the goods; after the user completes taking the goods, the cargo compartment door of the delivery robot is closed.

[0172] From the above, it can be seen that the target delivery order is moved to the target delivery order placement plate of the delivery robot according to the current placement position of the target delivery order in the target delivery order placement plate, so that the target delivery order is aligned with the grabbing position of the target delivery order placement plate; when the taking instruction of the target delivery order is received, the gripper is controlled to grab the target delivery order from the grabbing position to the cargo compartment outlet of the delivery robot; in this way, when multiple orders are mixed and placed, the target delivery order can be accurately controlled to move to the cargo compartment outlet, so that multiple orders can be mixed and placed on the same delivery order placement plate, and the cargo compartment storage capacity is fully utilized; when the target delivery order is at the cargo compartment outlet, the cargo compartment door of the delivery robot is opened, so that the target delivery order can be taken by the user, and the problem of other orders being taken by mistake is avoided. It can be seen that the order can be mixed and placed by the above-mentioned cargo compartment door control method, the utilization rate of the robot cargo compartment is improved, and the overall delivery efficiency of the robot is improved.

[0173] Those skilled in the art can understand that all or part of the steps in the above-mentioned delivery robot cargo compartment door control method can be completed by instructions, or by related hardware controlled by instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0174] Therefore, the embodiments of the present application provide a computer readable storage medium, which stores a plurality of computer programs. The computer programs can be loaded by a processor to execute any of the delivery robot cargo compartment door control methods provided by the embodiments of the present application. For example, the computer programs can be loaded by the processor to execute the following steps:

[0175] determining a target delivery order placement plate in which a target delivery order of a delivery robot is located; obtaining a current placement position of the target delivery order in the target delivery order placement plate; controlling the target delivery order placement plate to move according to the current placement position, so that the target delivery order is aligned with a grabbing position of the target delivery order placement plate; when a taking instruction of the target delivery order is received, controlling the gripper to grab the target delivery order from the grabbing position to a cargo compartment outlet of the delivery robot; and when the target delivery order is at the cargo compartment outlet, controlling a cargo compartment door of the delivery robot to be opened.

[0176] In some embodiments, the computer programs can be loaded by the processor to execute the following steps:

[0177] After the target goods placing plate is completed, an image of the target goods placing plate after the distribution is obtained; the image after the distribution is segmented to obtain each goods segmentation image in the image after the distribution; in response to an order association instruction of each goods segmentation image, the matched order of each goods segmentation image is associated with the target goods placing plate and recorded in an order placing database; and a goods placing plate associated with the target distribution order is queried from the order placing database to serve as a target goods placing plate of the target distribution order.

[0178] In some embodiments, the computer program can be loaded by the processor to perform the following steps:

[0179] The each goods segmentation image is associated with the matched order of the each goods segmentation image and recorded in the order placing database; when a pickup instruction of the target distribution order is received, an order comparison image of the grabbing position is obtained; a goods segmentation image associated with the target distribution order is queried from the order placing database to serve as an order reference image of the target distribution order; and if a similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, the gripper is controlled to grab the target distribution order from the grabbing position to an egress of a cargo compartment of the delivery robot.

[0180] In some embodiments, the computer program can be loaded by the processor to perform the following steps:

[0181] Based on the each goods segmentation image, a classification processing is performed to obtain an article category of the each goods segmentation image;

[0182] Order information of each distribution order of the delivery robot is obtained; based on the order information of each distribution order and the article category of the each goods segmentation image, a preliminary matched order of the each goods segmentation image is obtained; in response to a confirmation operation of the preliminary matched order of a first goods segmentation image in the each goods segmentation image, the preliminary matched order of the first goods segmentation image is associated with the first goods segmentation image and recorded in an order placing database.

[0183] In some embodiments, the computer program can be loaded by the processor to perform the following steps:

[0184] Based on the each goods segmentation image, a classification processing is performed to obtain an article category of the each goods segmentation image;

[0185] Obtain order information of each delivery order of the delivery robot; match the order information of each delivery order with the article categories of each article segmentation graph to obtain preliminary paired orders of each article segmentation graph; and in response to a modification operation of the preliminary paired orders of a second article segmentation graph, record the modified paired orders of the second article segmentation graph and the second article segmentation graph to the order placement database.

[0186] In some embodiments, the computer program can be loaded by the processor to perform the following steps:

[0187] Obtain order information of each delivery order of the delivery robot; match the order information of each delivery order with the article categories of each article segmentation graph to obtain preliminary paired orders of each article segmentation graph; and in response to a modification operation of the preliminary paired orders of a second article segmentation graph, record the modified paired orders of the second article segmentation graph and the second article segmentation graph to the order placement database.

[0188] In some embodiments, the computer program can be loaded by the processor to perform the following steps: based on the after-shipment image, identify the positions of the paired orders of each article segmentation graph to obtain the order placement positions of each article segmentation graph; record the order placement positions of each article segmentation graph and the paired orders of each article segmentation graph in the order placement database; and query the order placement positions recorded in association with the target delivery order from the order placement database to obtain the current placement position of the target delivery order.

[0189] In some embodiments, the computer program can be loaded by the processor to perform the following steps:

[0190] Detect whether the last delivery order of the target delivery order has been picked up; and when it is detected that the last delivery order has been picked up, control the target placement board to move according to the current placement position so that the target delivery order is aligned with the grabbing position of the target placement board.

[0191] The computer readable storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0192] In the above delivery robot hatch control method, computer readable storage medium, and electronic device embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described computer readable storage medium, electronic device and its corresponding units and the beneficial effects brought by them can be referred to the description of the delivery robot hatch control method in the above embodiments, and will not be described here in detail.

[0193] The above describes in detail a delivery robot hatch control method, an electronic device, and a computer readable storage medium provided by the embodiments of the present application. The principles and implementation manners of the present application are described by applying specific examples. The above embodiment descriptions are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges will be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for controlling the door of a delivery robot, characterized in that, The delivery robot is equipped with a cargo compartment including multiple storage panels and a gripper, and the method includes: Determine the target shelf where the delivery robot's target delivery order is located; Obtain the current placement position of the target delivery order on the target shelf; The target shelf is moved according to the current placement position so that the target delivery order is aligned with the grab position of the target shelf; Upon receiving the pickup instruction for the target delivery order, the gripper is controlled to pick up the target delivery order from the gripping position and move it to the cargo compartment exit of the delivery robot; When the target delivery order is at the cargo hold exit, the cargo hold door of the delivery robot is opened. The method further includes: After the target shelf is loaded with goods, obtain an image of the target shelf after loading. The sorted image is segmented to obtain a segmentation image of each item in the sorted image. In response to the order association instruction of each item segmentation image, the paired orders of each item segmentation image are associated with the target shelf and recorded in the order placement database; The method for determining the target shelf for the delivery robot's target delivery order includes: From the order placement database, query the storage board associated with the target delivery order to use as the target storage board for the target delivery order; The method further includes: Based on the image after order fulfillment, the location of the paired orders in each item segmentation image is identified to obtain the order placement location of each item segmentation image; The order placement positions of each item segmentation map are associated with the paired orders of each item segmentation map and recorded in the order placement database; The step of obtaining the current placement position of the target delivery order in the target shelf includes: From the order placement database, query the order placement location of the record associated with the target delivery order to use as the current placement location of the target delivery order.

2. The delivery robot door control method according to claim 1, characterized in that, The method further includes: The item segmentation diagrams and their paired orders are associated and recorded in the order placement database; Upon receiving the pickup instruction for the target delivery order, controlling the gripper to pick up the target delivery order from the gripping position and move it to the cargo compartment exit of the delivery robot includes: Upon receiving the pickup instruction for the target delivery order, obtain the order comparison image at the capture location; From the order placement database, query the item segmentation diagram of the record associated with the target delivery order, as the order reference diagram of the target delivery order; If the similarity between the order comparison image and the order reference image is greater than a preset similarity threshold, then the gripper is controlled to grab the target delivery order from the grabbing position to the cargo compartment exit of the delivery robot.

3. The delivery robot door control method according to claim 2, characterized in that, The step of associating the item segmentation images with their paired orders and recording them in the order placement database includes: Based on the item segmentation images, the item categories of each item segmentation image are obtained; Obtain the order information for each delivery order of the delivery robot; Based on the order information of each delivery order, the item categories of each item segmentation map are matched to obtain the preliminary matched orders of each item segmentation map; In response to the confirmation operation of the preliminary matching order for the first item segmentation map in each item segmentation map, the preliminary matching order for the first item segmentation map is associated with the first item segmentation map and recorded in the order placement database.

4. The delivery robot door control method according to claim 2, characterized in that, The step of associating the item segmentation images with their paired orders and recording them in the order placement database includes: Based on the item segmentation images, the item categories of each item segmentation image are obtained; Obtain the order information for each delivery order of the delivery robot; Based on the order information of each delivery order, the item categories of each item segmentation map are matched to obtain the preliminary matched orders of each item segmentation map; In response to the modification operation of the initial paired order of the second item segmentation map in each item segmentation map, the modified paired order of the second item segmentation map is associated with the second item segmentation map and recorded in the order placement database.

5. The delivery robot door control method according to claim 2, characterized in that, The step of obtaining the current placement position of the target delivery order in the target shelf includes: The target shelf is photographed to obtain a current image of the target shelf; From the order placement database, query the item segmentation diagram of the record associated with the target delivery order, as the order reference diagram of the target delivery order; The current image is matched with the order reference image to obtain the image position of the target delivery order in the current image; Based on the image location and the preset image space transformation relationship, the current placement position of the target delivery order in the target shelf is determined.

6. The delivery robot door control method according to claim 1, characterized in that, The step of controlling the movement of the target shelf according to the current placement position, so that the target delivery order is aligned with the grab position of the target shelf, includes: Check whether the previous delivery order for the target delivery order has been picked up; When the previous delivery order is detected to have been picked up, the target shelf is moved according to the current placement position so that the target delivery order is aligned with the grab position of the target shelf.

7. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the delivery robot door control method as described in any one of claims 1 to 6 when it invokes the computer program in the memory.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the delivery robot door control method according to any one of claims 1 to 6.

Images