Control method and device of robot storage bin and robot

Abstract

This application relates to the field of robotics technology and discloses a control method, device, and robot for a robot storage compartment. The control method includes: controlling the robot to move to a designated location; acquiring a user's clothing information and determining whether the user has permission to operate the storage compartment based on the clothing information; if the user has permission to operate the storage compartment, acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information. By acquiring the user's clothing information, determining whether the user has permission to operate the storage compartment based on the clothing information, and if the user has permission to operate the storage compartment, acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information, this application can achieve contactless control of the robot's storage compartment.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a control method, apparatus, and robot for a robotic storage compartment. Background Technology

[0002] Robots are a common term for automated machines, encompassing all machines that mimic human behavior or thought, as well as those that mimic other living beings. With the rapid development of artificial intelligence, robots are increasingly undertaking delivery tasks.

[0003] In existing robot delivery systems, users typically control the opening and closing of the robot's storage compartment by entering a password on the robot's display screen. However, with the global pandemic raging, if multiple users touch the same robot's display screen without timely and thorough disinfection of the robot, it increases the risk of users contracting the virus. Summary of the Invention

[0004] This application provides a control method, device, and robot for a robot storage compartment, which enables contactless control of the robot's storage compartment.

[0005] The embodiments of this application provide the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a control method for a robot storage compartment, the robot including the storage compartment, the control method for the robot storage compartment including:

[0007] Control the robot to move to the designated location;

[0008] Obtain the user's clothing information and determine whether the user has permission to operate the storage repository based on the clothing information;

[0009] If the user has permission to operate the storage container, the user's gesture information is obtained, and the storage container is controlled according to the operation intention indicated by the gesture information.

[0010] In some embodiments, the robot includes a camera device, and the preset verification information includes first information and second information, wherein the first information is clothing information and the second information is scene environment information;

[0011] Obtain the user's clothing information and determine whether the user has permission to operate the storage repository based on the clothing information, including:

[0012] The robot collects environmental information about its current location using a camera device.

[0013] When the scene environment information of the robot's current location matches the scene environment information in the preset verification information, the user's clothing information is obtained;

[0014] When a user's clothing information matches the clothing information in the preset verification information, it is determined that the user has the operation permission to operate the storage warehouse;

[0015] If the user's clothing information does not match the clothing information in the preset verification information, it is determined that the user does not have the operation permission to operate the storage container.

[0016] In some embodiments, clothing information includes clothing color information and clothing style information;

[0017] Obtaining the user's clothing information, determining whether the user has permission to operate the storage repository based on the clothing information, and also including:

[0018] The robot collects information about the color and style of the user's clothing using a camera device.

[0019] When the user's clothing color and style information match the clothing information in the preset verification information, it is determined that the user has the operation permission to operate the storage warehouse.

[0020] Otherwise, the user does not have permission to operate the storage repository.

[0021] In some embodiments, before controlling the robot to move to a designated location, the method includes:

[0022] Notify target users to upload clothing information to the robot via images and / or videos for personal verification;

[0023] Upon receiving clothing information uploaded by the target user, the received clothing information is recorded as the first piece of information in the preset verification information.

[0024] In some embodiments, before controlling the robot to move to a designated location, the method further includes:

[0025] Notify the target user to upload scene environment information for scene verification to the robot via images and / or videos;

[0026] Upon receiving scene environment information uploaded by the target user, the scene environment information is recorded as the second piece of information in the preset verification information.

[0027] In some embodiments, acquiring user gesture information and controlling the storage container according to the operation intention indicated by the gesture information includes:

[0028] The robot uses a camera device to capture gesture information from users with access to the storage compartment.

[0029] The corresponding operation intent is determined based on the user's gesture information, including opening the storage compartment door and closing the storage compartment door;

[0030] When the storage compartment door is closed and the intention is to open the storage compartment door, control the opening of the storage compartment door;

[0031] When the storage compartment door is open and the intention is to close the storage compartment door, control the closing of the storage compartment door.

[0032] In some embodiments, if the user has permission to operate the storage container, the method includes obtaining the user's gesture information, controlling the storage container according to the operation intention indicated by the gesture information, and then:

[0033] Obtain the user's gesture information;

[0034] Based on the intention of the robot to operate as indicated by the gesture information, control the robot to perform the corresponding operating actions.

[0035] In some embodiments, obtaining a user's clothing information and determining whether the user has permission to operate the storage repository based on the user's clothing information further includes:

[0036] Obtain the user's clothing information;

[0037] If the user's clothing information does not match the clothing information in the preset verification information, and the preset verification information is empty, it is determined that the user has the operation permission to operate the storage repository.

[0038] Secondly, embodiments of this application provide a control device for a robot storage compartment. The robot includes a storage compartment, and the control device for the robot storage compartment includes:

[0039] The navigation unit is used to control the robot to move to a designated location;

[0040] The acquisition unit is used to acquire the user's clothing information and determine whether the user has the operation permission to operate the storage warehouse based on the user's clothing information;

[0041] The control unit is used to acquire the user's gesture information if the user has the permission to operate the storage compartment, and control the storage compartment based on the operation intention indicated by the gesture information.

[0042] Thirdly, embodiments of this application provide a robot, including:

[0043] At least one processor; and

[0044] A memory that is communicatively connected to at least one processor; wherein,

[0045] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform a control method for the robotic storage compartment as described in the first aspect.

[0046] The beneficial effects of this application's embodiments are as follows: Unlike existing technologies, this application provides a control method for a robot storage compartment, including: controlling a robot to move to a designated location; acquiring a user's clothing information and determining whether the user has permission to operate the storage compartment based on the clothing information; if the user has permission to operate the storage compartment, acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information. By acquiring the user's clothing information, determining whether the user has permission to operate the storage compartment based on the clothing information, and if the user has permission to operate the storage compartment, acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information, this application can achieve contactless control of the robot's storage compartment. Attached Figure Description

[0047] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0048] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart illustrating a control method for a robotic storage bin provided in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of an environment for robot delivery provided in an embodiment of this application;

[0051] Figure 4 yes Figure 2 A detailed flowchart of step S202 in the process;

[0052] Figure 5 This is a flowchart illustrating a method for a target user to upload scene environment information, as provided in an embodiment of this application.

[0053] Figure 6 This is a flowchart illustrating a method for determining whether scene environment information of the robot's current location collected by the robot's camera device matches scene environment information in preset verification information, according to an embodiment of this application.

[0054] Figure 7This is a flowchart illustrating a method for determining whether a user has permission to operate a storage repository, as provided in an embodiment of this application.

[0055] Figure 8 This is a flowchart illustrating a method for a target user to upload clothing information, as provided in an embodiment of this application.

[0056] Figure 9 This is a flowchart illustrating another method for determining a user's permission to operate a storage container, provided in an embodiment of this application.

[0057] Figure 10 yes Figure 2 A detailed flowchart of step S203 in the process;

[0058] Figure 11 This is a schematic diagram of a process for controlling a robot to perform corresponding operating actions, provided in an embodiment of this application;

[0059] Figure 12 This is an interaction timing diagram of a robot, a server, and a mobile terminal provided in an embodiment of this application;

[0060] Figure 13 This is a schematic diagram of the structure of a control device for a robot storage bin provided in an embodiment of this application;

[0061] Figure 14 This is a schematic diagram of the structure of a robot provided in an embodiment of this application.

[0062] Explanation of icon numbers:

[0063] label name label name 10 robot 132 Acquisition Unit 20 server 133 Control Unit 30 mobile terminal 140 robot 130 Control device for robotic storage compartment 141 processor 131 Navigation unit 142 memory Detailed Implementation

[0064] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as "connected to" another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this specification are for illustrative purposes only.

[0065] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0066] The technical solution of this application will be described in detail below with reference to the accompanying drawings:

[0067] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0068] like Figure 1 As shown, the application environment 100 includes: a robot 10, a server 20, and a mobile terminal 30. The robot 10, server 20, and mobile terminal 30 are connected to each other via network communication. This network includes wired and / or wireless networks. It is understood that the network includes wireless networks such as 2G, 3G, 4G, 5G, Wi-Fi, and Bluetooth, and may also include wired networks such as serial cables and Ethernet cables.

[0069] In this embodiment, the robot 10 includes a mobile robot, such as a delivery robot, a transport robot, a warehousing robot, or a robot equipped with a storage compartment. The robot 10 includes a main body, drive wheels, a camera device, a lidar unit, a communication module, and a controller. The main body can be generally elliptical, triangular, D-shaped, or other shapes. The controller and storage compartment are located on the main body, and the drive wheels are mounted on the main body for driving the robot's movement.

[0070] In this embodiment, the drive wheel component includes a left drive wheel, a right drive wheel, and an omnidirectional wheel. The left and right drive wheels are respectively mounted on opposite sides of the main body. The omnidirectional wheel is mounted at the front of the bottom of the main body and is a movable caster wheel that can rotate 360 ​​degrees horizontally, allowing the robot to turn flexibly. The mounting of the left drive wheel, right drive wheel, and omnidirectional wheel forms a triangle to improve the stability of the robot's movement.

[0071] In this embodiment, a camera device is mounted on the robot's body and is used to acquire image data and / or video data. The camera device is communicatively connected to a controller and is used to acquire image data and / or video data within the camera's coverage area. For example, it can acquire image data and / or video data within a confined space and / or open space, or it can acquire image data and / or video data including user clothing information and / or gesture information, and then send the acquired image data and / or video data to the controller. In this embodiment, the camera device includes, but is not limited to, infrared cameras, night vision cameras, webcams, digital cameras, high-definition cameras, 4K cameras, 8K high-definition cameras, etc.

[0072] In this embodiment, the lidar is connected to the controller via communication. The lidar is mounted on the robot's body, for example, on the robot's mobile chassis. The lidar is used to acquire laser point cloud data. Specifically, the lidar acquires laser point cloud data within its monitoring range. The robot's mobile chassis is equipped with a communication module, and the laser point cloud data acquired by the lidar is transmitted to the controller via the communication module. In this embodiment, the lidar includes pulse lidar, continuous wave lidar, and other types of radar, and the mobile chassis includes all-purpose chassis, arched mobile chassis, and other robot mobile chassis.

[0073] In this embodiment, the communication module connects the mobile terminal and the server, and is used to receive data sent by the mobile terminal and the server, such as receiving clothing information and scene environment information of the target user sent by the server; or, to send data to the mobile terminal and the server, such as sending location information to the server. In this embodiment, the communication module can communicate with the Internet, and the communication module includes, but is not limited to, communication units such as WIFI module, ZigBee module, NB-IoT module, 4G module, 5G module, and Bluetooth module.

[0074] In this embodiment, the controller is located inside the main body and is electrically connected to the left drive wheel, right drive wheel, and omnidirectional wheel. As the robot's control core, the controller is used to control the robot's movement to a designated location and to perform some business logic processing. For example, the controller receives image data and / or video data sent by the camera unit and receives laser point cloud data sent by the LiDAR, and constructs an environmental map based on the laser point cloud data. Specifically, the controller uses Simultaneous Localization and Mapping (SLAM), i.e., the laser SLAM algorithm, to calculate the laser point cloud data of the monitored area to construct the environmental map. In this embodiment, the laser SLAM algorithm includes Kalman filtering, particle filtering, and graph optimization methods.

[0075] In the embodiments of this application, the controller can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a microcontroller, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. The controller can also be any conventional processor, controller, microcontroller, or state machine. The controller can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration, or one or more combinations of a microcontroller unit (MCU), a field-programmable gate array (FPGA), and a system-on-chip (SoC).

[0076] It is understood that the robot 10 in this application embodiment also includes a storage module, which includes, but is not limited to, one or more of the following devices: FLASH flash memory, NAND flash memory, vertical NAND flash memory (VNAND), NOR flash memory, resistive random access memory (RRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), spin-transfer torque random access memory (STT-RAM).

[0077] In this embodiment of the application, during the operation of the robot 10, the controller uses Simultaneous Localization and Mapping (SLAM), namely laser SLAM algorithm, for localization and navigation, and constructs a map and localizes based on environmental data.

[0078] In this embodiment, the server 20 communicates with the robot 10 and the mobile terminal 30, and is used to send clothing information and / or scene environment information uploaded by the target user to the robot 10, or to send gesture information for picking up objects and the picking time to the mobile terminal 30. The server 20 is an electronic device with data processing capabilities, which can be integrated into the robot 10. There can be multiple servers 20, and multiple servers can form a server cluster. For example, the server cluster includes: a first server, a second server, ..., an Nth server; or, the server cluster can be a cloud computing service center, which includes several servers. The servers in this embodiment include, but are not limited to: tower servers, rack servers, blade servers, and cloud servers. Preferably, the server is a cloud server (Elastic Compute Service, ECS).

[0079] In this embodiment, the mobile terminal 30 is communicatively connected to the server 20 and is used to send clothing information of the target user and / or scene environment information collected by the target user to the server 20, or to receive gesture information for retrieving an object and the time of retrieval sent by the server 20. The mobile terminal 30 is equipped with an application (APP), through which the user can send clothing information of the target user and / or scene environment information collected by the target user to the server 20. The mobile terminal 30 includes, but is not limited to, mobile communication devices, mobile personal computer devices, portable entertainment devices, or other electronic devices with video playback and internet access capabilities.

[0080] Please see Figure 2 , Figure 2 This is a flowchart illustrating a control method for a robotic storage bin provided in an embodiment of this application;

[0081] The control method for the robot storage compartment is applied to a robot, such as a mobile robot, which includes a storage compartment. Specifically, the controller of the control method for the robot storage compartment is one or more processors of the robot.

[0082] like Figure 2 As shown, the control method for the robot's storage compartment includes:

[0083] Step S201: Control the robot to move to the designated position;

[0084] Specifically, the robot receives delivery instructions from the administrator and controls the robot to move to the designated location according to the delivery instructions. The delivery instructions include the robot's initial position and the designated location that the hotel robot wants to reach.

[0085] Please refer to the following: Figure 3 , Figure 3 This is a schematic diagram of an environment for robot delivery provided in an embodiment of this application;

[0086] like Figure 3 As shown, the robot's delivery environment includes obstacles. The robot corresponds to an initial position and a designated position. Based on the initial position and the designated position, combined with the obstacles, an original path and a second path can be planned.

[0087] In this embodiment, the robot receives a delivery instruction input by the administrator and controls the robot to move to a designated location according to the delivery instruction, including:

[0088] Get the global map;

[0089] Based on the global map, multiple sub-paths are planned along the direction of travel between the robot's current position and the target position that the robot wants to reach.

[0090] The shortest path among the multiple sub-paths is taken as the original path. Based on this original path, the robot is controlled to start from the current position and move along the direction of the original path.

[0091] Specifically, the global map includes an environmental map of the robot's environment, such as a global SLAM map. This map can be obtained by the robot from a server or stored in its memory by the robot's controller. The direction of travel, or the robot's forward direction, is detected by an accelerometer sensor mounted on the robot's body. This accelerometer sensor is also used to detect the robot's speed and acceleration.

[0092] Specifically, based on the global map, the robot's current position and the target position to be reached are determined. Between the current position and the target position, multiple sub-paths are planned along the robot's direction of travel. Each sub-path starts from the current position and points to the target position. The path with the shortest distance among the multiple sub-paths is taken as the original path. Based on the original path, the robot is controlled to start from the current position and travel along the direction of the original path.

[0093] It is understandable that robots will pass through public areas during delivery. Public areas have fixed hardware facilities and people, which can hinder the robot's movement. Therefore, the robot needs to use sensors and LiDAR installed on its body to detect whether it encounters obstacles in real time. If it encounters an obstacle, it needs to replan its path.

[0094] In this embodiment of the application, controlling the robot to move to a designated location further includes:

[0095] Detect whether there are obstacles around the robot within a preset range;

[0096] If there are obstacles within the preset range, the robot will be controlled to plan a second path to avoid the obstacles.

[0097] Based on the second path, control the robot to travel along the second path to reach the designated location.

[0098] Specifically, the preset range is a circular area with the robot as the center and a preset distance as the radius. The preset distance can be 1m or set by the administrator according to the actual situation. When the robot's sensors detect an obstacle within the preset range, they send the detection information to the controller. The controller then plans a second path between the robot's current position and the target position that the robot wants to reach. The second path is different from the original path. Then, based on the second path, the robot is controlled to move to the designated position.

[0099] In this embodiment, the robot detects in real time whether there are obstacles around it within a preset range. If an obstacle is detected, the robot replans its path to avoid it, thus preventing the robot from colliding with the obstacle and being damaged, thereby improving the robot's safety.

[0100] Step S202: Obtain the user's clothing information and determine whether the user has the permission to operate the storage container based on the user's clothing information;

[0101] Please see Figure 4 , Figure 4 yes Figure 2 A detailed flowchart of step S202 in the process;

[0102] like Figure 4 As shown, the robot includes a camera device to acquire the user's clothing information and determine whether the user has permission to operate the storage compartment based on the clothing information, including:

[0103] Step S2021: Collect scene environment information of the robot's current location using the camera device configured on the robot;

[0104] Specifically, the scene environment information includes scene environment video. After the robot moves to the designated position, the robot's controller controls the camera device to capture scene environment video of the robot's current location. It can be understood that in order to obtain scene environment video of the robot's location more comprehensively, the robot's controller can control the robot to rotate 360 ​​degrees horizontally to capture scene environment video of the robot's location, or control the camera device to rotate to capture scene environment video from different angles.

[0105] Alternatively, the scene environment information may also include scene environment images. The robot's camera device takes a predetermined number of scene environment images within a predetermined time. The predetermined time can be 1 minute, or the administrator can determine the specific time based on the actual situation and pre-set it in the robot's controller. The predetermined number of images can be 50, or the administrator can determine the specific number based on the actual situation and pre-set it in the robot's controller.

[0106] Furthermore, after the robot's camera device collects the scene environment information of the robot's current location, the robot's controller determines whether the scene environment information collected by the robot's camera device matches the scene environment information in the preset verification information.

[0107] In this embodiment of the application, the preset verification information includes first information and second information, wherein the first information is clothing information and the second information is scene environment information;

[0108] Please see Figure 5 , Figure 5 This is a flowchart illustrating a method for a target user to upload scene environment information, as provided in an embodiment of this application.

[0109] like Figure 5 As shown, before controlling the robot to move to the designated position, the method also includes:

[0110] Step S501: Notify the target user to upload scene environment information for scene verification to the robot via images and / or videos;

[0111] Specifically, the robot includes a touch screen. After the administrator places the items needed by the target user into the robot's storage compartment, they edit a notification message on the robot's touch screen and send the notification message to the cloud server through the robot's controller. The notification message includes a scene environment information upload notification and the target user's phone number. After receiving the notification message, the cloud server identifies the target user's phone number in the notification message and sends an environment information upload notification to the target user's mobile terminal corresponding to that phone number, so as to notify the target user to upload scene environment information for scene verification through pictures and / or videos.

[0112] Furthermore, after receiving the environmental information upload notification, the target user can use the camera on their mobile terminal to take pictures and / or videos of the designated location where the robot needs to deliver the items. After taking the pictures and / or videos, the target user can upload them to the cloud server through the APP on their mobile terminal or the web link in the environmental information upload notification, and the cloud server will then forward them to the robot.

[0113] Understandably, in order to improve the speed of scene verification, the scene environment information used for scene verification can be pre-shot and stored in the robot's storage unit by the administrator, or it can be captured by the monitoring camera in the monitoring system at the designated location that the robot needs to move to and uploaded to the cloud server, and then forwarded to the robot by the cloud server.

[0114] Step S502: Upon receiving the scene environment information uploaded by the target user, record the scene environment information as the second piece of information in the preset verification information.

[0115] Specifically, after receiving the image and / or video uploaded by the target user showing the designated location where the robot needs to deliver the item, the cloud server marks it as scene environment information and forwards the scene environment information to the robot. After receiving the scene environment information, the robot's controller records it as the second piece of information in the preset verification information and stores it in the robot's storage unit.

[0116] Please see Figure 6 , Figure 6 This is a flowchart illustrating a method for determining whether scene environment information of the robot's current location collected by the robot's camera device matches scene environment information in preset verification information, according to an embodiment of this application.

[0117] like Figure 6 As shown, the method for determining whether the scene environment information of the robot's current location collected by the robot's camera device matches the scene environment information in the preset verification information includes:

[0118] Step S601: Determine whether the type of scene environment information of the robot's current location collected by the robot's camera device is scene environment video;

[0119] Specifically, the types of scene environment information collected by the robot's camera device at the robot's current location include scene environment video and scene environment image. If the type of scene environment information collected by the robot's camera device at the robot's current location is scene environment video, then proceed to step S602. If the type of scene environment information collected by the robot's camera device at the robot's current location is not scene environment video, that is, if the type of scene environment information collected by the robot's camera device at the robot's current location is scene environment image, then proceed to step S604.

[0120] Step S602: Perform unified transcoding on the first video and the second video;

[0121] Specifically, the first video is a scene environment video of the robot's current location captured by the robot's camera device, and the second video is a scene environment video from the scene environment information in the preset verification information. It can be understood that the preset verification information is stored in the robot's storage unit. The first video and the second video are uniformly transcoded to convert them into the same encoding format for subsequent processing. The transcoding method used in this application embodiment includes, but is not limited to, using FFmpeg software to uniformly convert the first video and the second video into MP4 format.

[0122] Step S603: Perform frame extraction operations on the first video and the second video respectively to obtain the first image and the second image;

[0123] Specifically, a certain number of frame images are extracted per second based on the video frame rate as the first image and the second image. For example, when the frame rate of the first or second video is 24 frames per second, an average of 3 images can be extracted per second as comparison images. All images obtained from the frame extraction operation on the first video are used as the first image, and all images obtained from the frame extraction operation on the second video are used as the second image. It is understood that if the frame rate is increased, more images can be extracted. This application does not limit the specific methods and rules for image extraction.

[0124] Step S604: Perform similarity comparison processing on the first image and the second image to obtain the first comparison similarity;

[0125] Specifically, the first and second images are first converted to grayscale images through grayscale processing, and then reduced to a predetermined size to obtain the first and second converted images. This step can compress the amount of information in the image and optimize processing efficiency. Converting to grayscale reduces the information of each pixel from three dimensions (red, green, and blue) to one dimension (grayscale value). Compression reduces the number of pixels in the image to a manageable level. It can be understood that in this embodiment, the image can be converted to 64 levels of grayscale, that is, divided into 64 levels from absolute white to absolute black, with the color value of each pixel between 0 and 63. The image is then compressed to an 8×8 pixel size, containing a total of 64 pixels. This number of pixels can be set by the administrator according to actual needs. This value determines the number of bits in the generated hash value, i.e., the hash fingerprint. 64 pixels correspond to a 64-bit hash fingerprint. For example, if set to 16×16 pixels, a 256-bit hash fingerprint will be generated. Similarly, a higher value improves accuracy but also increases processing complexity.

[0126] Then, the first and second converted images are processed into black and white respectively. For example, for a single image, the average gray value of all its pixels is calculated, and then it is determined whether the gray value of each pixel is greater than the average gray value of all pixels. If the result is true, the pixel is marked as "black" and assigned a value of 1. If the result is false, the pixel is marked as "white" and assigned a value of 0. Then, the values ​​of each pixel in the converted black and white image are arranged in order from left to right and from top to bottom to obtain a 64-bit hash value composed of 0 and 1, which is the hash fingerprint of the corresponding image.

[0127] Next, the Hamming distance between the first and second hash fingerprints is calculated. The two hash fingerprints are compared bit by bit to find the similarities and differences. If the comparison result shows the number of identical bits, for example, a Hamming distance of 58 means that the hash fingerprints of the two images have 58 identical bits out of 64. The first comparison similarity is then obtained by dividing the Hamming distance by the total number of bits in the hash fingerprint. For example, if the first comparison similarity is 58 / 64 = 0.90625, it means that the similarity between the first and second images is 90.625% after this method.

[0128] It is understood that the first image can be obtained by extracting frames from the first video, or it can be a scene environment image captured by the robot's camera device. The second image can be obtained by extracting frames from the second video, or it can be a scene environment image stored in the preset verification information in the robot's storage unit.

[0129] Step S2022: When the scene environment information of the robot's current location matches the scene environment information in the preset verification information, obtain the user's clothing information;

[0130] Specifically, it is determined whether the first comparison similarity is greater than a preset similarity threshold. The preset similarity threshold is a similarity coefficient preset in the robot's memory. This coefficient reflects the degree of similarity between two images and is used to determine whether the second image is substantially similar to the first image when the similarity between the two images reaches this threshold. The preset similarity threshold can be set to, for example, 0.9, which means that when the two images are 90% similar, the second image is determined to be substantially similar to the first image.

[0131] Understandably, if the first comparison similarity is greater than or equal to the preset similarity threshold, it is determined that the scene environment information of the robot's current location matches the scene environment information in the preset verification information, that is, the robot's current location is the designated location that the robot should reach. At this time, the robot's controller controls the camera device to obtain the user's clothing information.

[0132] Alternatively, if the first comparison similarity is less than the preset similarity threshold, it is determined that the scene environment information of the robot's current location does not match the scene environment information in the preset verification information. That is, the robot's current location is not the designated location that the robot should reach. In this case, the robot's controller controls the robot to re-plan the path so that the robot can move to the designated location to deliver items to the target user.

[0133] In this embodiment of the application, by obtaining the user's clothing information when the scene environment information of the robot's current location matches the scene environment information in the preset verification information, that is, when the robot arrives at the specified location, the user's identity verification process during delivery can be simplified.

[0134] Step S2023: When the user's clothing information matches the clothing information in the preset verification information, determine that the user has the operation permission to operate the storage container;

[0135] Specifically, when the robot moves to the designated location and verifies that the scene environment information of the robot's current location matches the scene environment information in the preset verification information, the camera device controlling the robot acquires the user's clothing information, which includes clothing color information and clothing style information.

[0136] For details, please refer to Figure 7 , Figure 7 This is a flowchart illustrating a method for determining whether a user has permission to operate a storage repository, as provided in an embodiment of this application.

[0137] like Figure 7 As shown, determining whether a user has permission to operate the storage repository includes:

[0138] Step S701: Collect the user's clothing color information and clothing style information through the camera device configured on the robot;

[0139] Specifically, when the robot's camera detects a person moving toward the robot's location, the robot controls the camera to capture video and / or images containing information about the person's clothing. This person is the first user. The duration of the video and the number of images captured by the robot's camera containing the person's clothing information can be preset in the robot's controller by the administrator according to the actual situation.

[0140] It is understandable that if the robot's camera device captures a video containing information about the user's clothing, then the video will be subjected to frame extraction to obtain an image containing the user's clothing information, which includes clothing color information and clothing style information.

[0141] Furthermore, the robot includes a clothing analysis module, which is used to determine similarity data for clothing color information and similarity data for clothing style information. The robot's camera device sends videos and / or images containing the user's clothing information to the robot's controller. After receiving the video, the robot's controller performs frame extraction to obtain an image containing the user's clothing information, extracts the user's clothing color information and clothing style information from the image, and then calls the clothing analysis module to compare the user's clothing color information with the clothing color information in the preset verification information to determine the similarity data for clothing color information. It also compares the user's clothing style information with the clothing style information in the preset verification information to determine the similarity of clothing style information. The clothing style information includes clothing type, for example: the top type can be a long-sleeved top, a short-sleeved top, a crew neck top, a U-neck top, a V-neck top; or the skirt type can be a long skirt, a short skirt; or the trouser type can be long trousers, shorts, etc.

[0142] In the embodiments of this application, the clothing analysis module includes, but is not limited to, electronic units with processing capabilities such as a central processing unit (CPU) and a microcontroller unit (MCU).

[0143] Please see Figure 8 , Figure 8 This is a flowchart illustrating a method for a target user to upload clothing information, as provided in an embodiment of this application.

[0144] like Figure 8 As shown, before controlling the robot to move to the designated position, the method also includes:

[0145] Step S801: Notify the target user to upload clothing information for personal authentication to the robot via images and / or videos;

[0146] Specifically, after the administrator places the items needed by the target user into the robot's storage compartment, they edit a notification message on the robot's touch screen and send the message to the cloud server via the robot's controller. The notification message includes a clothing information upload notification and the target user's phone number. Upon receiving the notification message, the cloud server identifies the target user's phone number and sends a clothing information upload notification to the target user's mobile terminal corresponding to that phone number, notifying the target user to upload clothing information for personal authentication via images and / or videos.

[0147] Furthermore, after receiving the clothing information upload notification, the target user can take pictures and / or videos containing their own clothing information using the camera of their mobile terminal. After taking the pictures and / or videos, the target user can upload them to the cloud server through the APP on the mobile terminal or the web link in the clothing information upload notification, and the cloud server will then send them to the robot.

[0148] Understandably, in order to improve the speed of user authentication, clothing information used for personal authentication can be collected by the target user at a designated location by a camera device and uploaded to a cloud server, which then forwards it to the robot.

[0149] Step S802: Upon receiving clothing information uploaded by the target user, record the received clothing information as the first piece of information in the preset verification information.

[0150] Specifically, after receiving an image and / or video containing clothing information uploaded by the target user, the cloud server marks it as clothing information and forwards it to the robot. After receiving the clothing information, the robot's controller records it as the first piece of information in the preset verification information and stores it in the storage unit.

[0151] Step S702: Determine whether the user's clothing color information and clothing style information match the clothing information in the preset verification information;

[0152] Specifically, it determines whether the user's clothing color information matches the clothing color information in the preset verification information, and whether the user's clothing style information matches the clothing style information in the preset verification information.

[0153] In this embodiment of the application, determining whether the user's clothing color information matches the clothing color information in the preset verification information includes:

[0154] Color histogram data is calculated based on the HSV color space, where H, S, and V represent hue, saturation, and value, respectively.

[0155] Based on the color histogram data and the Bach distance metric algorithm, similarity data of clothing color information is obtained;

[0156] If the similarity data of clothing color information is greater than the preset similarity data of clothing color information, then it is determined that the user's clothing color information matches the clothing color information in the preset verification information.

[0157] Specifically, color quantization is first performed, dividing the color space into several smaller color bins. Considering that illumination can affect color recognition, during the model training phase, the three components are quantized into h values ​​respectively. color Color range, s color Color ranges and v color For each color interval, calculate the number of pixels h(i) whose color falls within each interval, and then normalize the h(i) to obtain the color histogram data for different clothing items. The specific formula is shown below:

[0158]

[0159] N = h color +s color +v color (2)

[0160] Where H(i) represents the normalized number of pixels, N represents the total number of pixels, and h color s color v color These represent the number of pixels whose color falls within the corresponding color range.

[0161] Based on the obtained color histogram data, the Bach distance method is used to calculate the clothing color similarity data between the color histogram of the user's clothing information and the color histogram of the clothing information in the preset verification information. The specific formula is as follows:

[0162]

[0163] Among them, S clothes This represents data on the similarity of clothing colors.

[0164] If the similarity data of clothing color information is greater than the preset similarity data of clothing color information, then it is determined that the user's clothing color information matches the clothing color information in the clothing information in the preset verification information. The preset similarity data of clothing color information can be set by the administrator according to the actual situation.

[0165] In this embodiment of the application, determining whether the user's clothing style information matches the clothing style information in the preset verification information includes:

[0166] The robot generates clothing style similarity data based on the clothing style information collected by its camera device.

[0167] If the similarity data of clothing style information is greater than the preset similarity data of clothing style information, then it is determined that the user's clothing style information matches the clothing style information in the clothing information in the preset verification information.

[0168] Specifically, the robot's controller calls the clothing analysis module to identify the clothing style information in each image captured by the robot's camera device, generating multiple first and second identification results. The first identification result is the clothing type, such as: top, skirt, or pants. The second identification result is the specific style of the first identification result. For example, when the first identification result is a top, the second identification result can be a long-sleeved top, short-sleeved top, crew neck top, U-neck top, or V-neck top; or when the first identification result is a skirt, the second identification result can be a long skirt or short skirt; or when the first identification result is pants, the second identification result can be long pants or shorts, etc.

[0169] Understandably, the robot's clothing analysis module can also identify clothing style information in the clothing information in the preset verification information uploaded by the target user, and generate a first target identification result and a second target identification result. The first target identification result is the clothing type of the target user, and the second target identification result is the specific style of the first identification result.

[0170] Specifically, the robot's clothing analysis module compares multiple first identification results with the first target identification result. When the first identification result and the first target identification result are the same, i.e., the clothing type is the same, the first identification result is determined to be reliable. Then, it compares whether the second identification result corresponding to the reliable first identification result is the same as the second target identification result. When the second identification result corresponding to the reliable first identification result is the same as the second target identification result, i.e., the specific style of the clothing type is the same, the second identification result is determined to be reliable. Next, the proportion of reliable second identification results to the total number of second identification results is calculated. This proportion is the clothing style information similarity data. If the clothing style information similarity data is greater than the preset clothing style information similarity data, then the user's clothing style information is determined to match the clothing style information in the preset verification information. The preset clothing style information similarity data can be 90%, or it can be set by the administrator according to the actual situation.

[0171] Furthermore, if the user's clothing color information matches the clothing color information in the preset verification information, and the user's clothing style information matches the clothing style information in the preset verification information, proceed to step S703: determine that the user has the operation permission to operate the storage compartment; otherwise, proceed to step S704: determine that the user does not have the operation permission to operate the storage compartment.

[0172] Step S703: Determine if the user has permission to operate the storage container;

[0173] Specifically, if the user's clothing color and style information match the clothing information in the preset verification information, it indicates that the user is the target user who needs to retrieve the items delivered by the robot. At this time, the user has the right to control the robot's controller to operate the robot's storage compartment. The right to operate the storage compartment includes controlling the robot to open and / or close the storage compartment.

[0174] Step S704: Determine that the user does not have permission to operate the storage container.

[0175] Specifically, if the user's clothing color information does not match the clothing color information in the preset verification information, or if the user's clothing style information does not match the clothing style information in the preset verification information, it indicates that the user is not the target user who needs to pick up the items delivered by the robot. In this case, the robot's controller controls the robot to stay in place and re-controls the camera device to collect the clothing color information and clothing style information of the second user, proceeding to step S702, where the second user is different from the first user.

[0176] In this embodiment, the user's clothing information is obtained when the scene environment information of the robot's current location matches the scene environment information in the preset verification information. When the user's clothing color information and clothing style information both match the clothing information in the preset verification information, it is determined that the user has the operation permission to operate the storage compartment. This can more accurately verify the user's identity information, realize the delivery of items to specific users and protect the user's privacy.

[0177] Step S2024: If the user's clothing information does not match the clothing information in the preset verification information, determine that the user does not have the operation permission to operate the storage container.

[0178] Specifically, this step is implemented in the same way as step S704, and will not be described again here.

[0179] Please refer to the following: Figure 9 , Figure 9 This is a flowchart illustrating another method for determining a user's permission to operate a storage container, provided in an embodiment of this application.

[0180] like Figure 9 As shown, obtaining the user's clothing information and determining whether the user has permission to operate the storage repository based on the user's clothing information also includes:

[0181] Step S901: Obtain the user's clothing information;

[0182] Specifically, this step is implemented in the same way as step S701, and will not be described again here.

[0183] Step S902: When the user's clothing information does not match the clothing information in the preset verification information, and the preset verification information is empty, determine that the user has the operation permission to operate the storage container.

[0184] Understandably, during a global pandemic, the items delivered by robots can be specific medical supplies, and the target users can be specific groups, such as medical personnel or volunteers wearing protective suits. To improve the speed of robot identity verification and delivery efficiency, preset verification information can be initialized to an empty value before determining whether a user has permission to operate the storage compartment.

[0185] Specifically, after the robot completes its delivery to the previous user, its camera captures the user's waving gesture. Upon detecting this gesture, the robot's controller initializes the preset verification information to a null value. In other words, after completing the delivery for the previous user, the robot removes the operation permissions previously granted only to that user, allowing it to grant operation permissions to any user thereafter.

[0186] Alternatively, before the robot delivers medical supplies to a specific group, the administrator can initialize the preset verification information stored in the robot to an empty value.

[0187] Furthermore, after the robot's camera device acquires the clothing information of users belonging to a specific group, the robot's controller determines whether the user's clothing information matches the clothing information in the preset verification information. If the user's clothing information does not match the clothing information in the preset verification information, it is determined whether the preset verification information is empty. If the preset verification information is empty, it is determined that the user has the operation permission to operate the storage compartment.

[0188] In this embodiment of the application, by obtaining the user's clothing information, when the user's clothing information does not match the clothing information in the preset verification information and the preset verification information is empty, it is determined that the user has the operation permission to operate the storage warehouse. This can improve the speed of the robot to verify the identity of people in a specific group, simplify the process, and improve delivery efficiency.

[0189] Step S203: If the user has the permission to operate the storage container, then obtain the user's gesture information and control the storage container according to the operation intention indicated by the gesture information.

[0190] Specifically, if the user has the authority to operate the storage compartment, that is, the user is the target user who needs to retrieve the item, the robot's controller acquires the video and / or image containing the user's gesture information captured by the camera device, identifies the operation intention indicated by the gesture information, and controls the storage compartment according to the operation intention.

[0191] Please refer to the following: Figure 10 , Figure 10 yes Figure 2 A detailed flowchart of step S203 in the process;

[0192] like Figure 10 As shown, the system acquires the user's gesture information and controls the storage container based on the operation intent indicated by the gesture information, including:

[0193] Step S2031: Collect gesture information of users with operating permissions to operate the storage compartment through a camera device configured on the robot;

[0194] Specifically, after determining that the user has the right to operate the storage compartment, the robot's controller acquires video and / or images containing the user's gesture information captured by the camera device. If the robot's camera device captures a video containing the user's gesture information, then the video is subjected to frame extraction to obtain an image containing the user's gesture information, which includes OK gesture, scissor gesture, fist gesture, thumbs-up gesture, etc.

[0195] Step S2032: Determine the corresponding operation intention based on the user's gesture information, wherein the operation intention includes opening the storage compartment door and closing the storage compartment door;

[0196] Specifically, the corresponding operation intent is determined based on the user's gesture information, including:

[0197] Based on each image containing the user's gesture information, generate the user's gesture contour feature data corresponding to each image;

[0198] The user's gesture contour feature data is compared with the preset gesture contour feature data to determine the recognition result;

[0199] Based on the recognition results, the user's gesture information is used to determine the corresponding operation intention.

[0200] Specifically, the robot's controller processes each image containing the user's gesture information to obtain multiple gesture contours composed of sets of points. For each gesture contour, a loop traversal algorithm is used to compare the ordinates of each point in the set. The difference between the maximum and minimum ordinates is taken as the user's gesture contour feature data. The user's gesture contour feature data is compared with the preset gesture contour feature data corresponding to the preset gesture information stored in the robot. If the similarity between the two is greater than 90%, the recognition result is determined to be reliable; otherwise, the recognition result is unreliable. When the proportion of reliable recognition results to the total recognition results is greater than 90%, the operation intention corresponding to the preset gesture contour feature data of the reliable recognition result is determined. For example, if the preset gesture contour feature data corresponding to the reliable recognition result is the "OK" gesture contour feature data, the operation intention is to open the door of the robot's storage compartment; or, if the preset gesture contour feature data corresponding to the reliable recognition result is the "fist" gesture contour feature data, the operation intention is to close the door of the robot's storage compartment.

[0201] In this embodiment of the application, the method further includes:

[0202] After the administrator places the item into the robot's storage compartment, they can set preset gesture information through the robot's touch screen and establish a mapping relationship between the preset gesture outline feature data corresponding to the preset gesture information and its corresponding operation intention. This mapping relationship can be set by the administrator according to the actual situation.

[0203] Step S2033: When the storage compartment door is closed and the operation intention is to open the storage compartment door, control the opening of the storage compartment door;

[0204] Specifically, when the door of the robot's storage compartment is closed, meaning the user has not yet retrieved the item, and the user's gesture indicates an intention to open the storage compartment door (i.e., the user needs to retrieve the item), the robot's controller will open the storage compartment door.

[0205] Step S2034: When the storage compartment door is open and the operation intention is to close the storage compartment door, control the closing of the storage compartment door.

[0206] Specifically, when the robot's storage compartment door is open, meaning the user has taken out the required item and the user's intention is to close the storage compartment door, i.e., when the robot needs to close the storage compartment door to complete the delivery, the robot's controller will control the storage compartment door to close.

[0207] In this embodiment, by acquiring the user's gesture information and controlling the opening or closing of the storage compartment according to the operation intention indicated by the gesture information, it is possible to avoid the user touching the robot's touch screen, thereby achieving contactless control of the robot's storage compartment and reducing the risk of users contracting the virus during the pandemic.

[0208] Please refer to the following: Figure 11 , Figure 11 This is a schematic diagram of a process for controlling a robot to perform corresponding operating actions, provided in an embodiment of this application;

[0209] like Figure 11 As shown, if the user has permission to operate the storage container, the method involves obtaining the user's gesture information, controlling the storage container based on the operation intent indicated by the gesture information, and then including:

[0210] Step S1101: Obtain the user's gesture information;

[0211] Specifically, after controlling the robot's storage compartment according to the user's gesture information, the user's gesture information is retrieved again. The specific implementation method of retrieving the user's gesture information is the same as that of step S2031, and will not be repeated here.

[0212] Step S1102: Control the robot to perform the corresponding operation action according to the intention of the operation robot indicated by the gesture information.

[0213] Specifically, the intention of operating the robot includes controlling the robot to automatically return to its initial position. The robot's controller processes each image containing the user's gesture information to obtain the user's gesture contour feature data. It compares the user's gesture contour feature data with the preset gesture contour feature data corresponding to the preset gesture information stored in the robot. If the similarity between the two is greater than 90%, the recognition result is determined to be reliable; otherwise, the recognition result is unreliable. When the proportion of reliable recognition results to the total recognition results is greater than 90%, the intention of operating the robot corresponding to the preset gesture contour feature data of the reliable recognition result is determined. For example, if the preset gesture contour feature data corresponding to the reliable recognition result is scissor gesture contour feature data, then the intention of operating the robot is to control the robot to automatically return to its initial position. At this time, the robot's controller controls the robot to perform the corresponding operation action, that is, to control the robot to automatically return to its initial position.

[0214] Please refer to the following: Figure 12 , Figure 12 This is an interaction timing diagram of a robot, a server, and a mobile terminal provided in an embodiment of this application;

[0215] like Figure 12 As shown, the interaction process between the robot, the server, and the mobile terminal includes:

[0216] Step S1201: Send notification information;

[0217] Specifically, the robot sends notification information to the server, which includes notifications of scene environment information upload, clothing information upload, and the target user's phone number.

[0218] Step S1202: Receive notification information and identify the target user's phone number;

[0219] Specifically, the server receives notification information sent by the robot and identifies the target user's phone number in the notification information to send environmental information upload notifications and clothing information upload notifications to the mobile terminal of the target user corresponding to that phone number.

[0220] Step S1203: Send notification information;

[0221] Specifically, the server sends environmental information upload notifications and clothing information upload notifications to the target user's mobile terminal, instructing the target user to upload scene environment information for scene verification and clothing information for personal identity verification via images and / or videos.

[0222] Step S1204: Receive notification information;

[0223] Specifically, the target user's mobile terminal receives notification information sent by the server, which includes notifications for uploading environmental information and clothing information.

[0224] Step S1205: Upload preset verification information;

[0225] Specifically, the target user uploads preset verification information to the server via the APP on the mobile terminal or the webpage link in the clothing information upload notification and the webpage link in the environment information upload notification. The preset verification information includes scene environment information of the designated location where the robot needs to deliver the goods and the target user's clothing information.

[0226] Step S1206: Send preset verification information;

[0227] Specifically, after receiving the preset verification information uploaded by the mobile terminal, the server sends it to the robot.

[0228] Step S1207: After receiving the preset verification information and controlling the robot to move to the designated position, obtain the user's clothing information and determine whether the user has the operation permission to operate the storage compartment based on the user's clothing information;

[0229] Specifically, the robot receives and stores preset verification information; after the robot's controller moves the robot to a designated location, the robot's camera captures video and / or images containing the user's clothing information, including clothing color information and clothing style information. When the user's clothing color information matches the clothing color information in the preset verification information, and the user's clothing style information matches the clothing style information in the preset verification information, it is determined that the user has the operation permission to operate the storage compartment.

[0230] Step S1208: If the user has permission to operate the storage container, obtain the user's gesture information and control the storage container according to the operation intention indicated by the gesture information.

[0231] Specifically, after determining that the user has the right to operate the storage compartment, the robot's controller acquires video and / or images containing the user's gesture information captured by the camera device. Based on each image containing the user's gesture information, it generates gesture contour feature data corresponding to each image. The user's gesture contour feature data is compared with preset gesture contour feature data to determine the recognition result. Based on the recognition result, the operation intention corresponding to the user's gesture information is determined. The operation intention includes opening the storage compartment door and closing the storage compartment door.

[0232] In this embodiment, a method for controlling a robot storage compartment is provided, comprising: controlling a robot to move to a designated location; acquiring a user's clothing information and determining whether the user has permission to operate the storage compartment based on the clothing information; if the user has permission to operate the storage compartment, acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information. By acquiring the user's clothing information, determining whether the user has permission to operate the storage compartment based on the clothing information, and if the user has permission to operate the storage compartment, acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information, this application can achieve contactless control of the robot's storage compartment.

[0233] Please refer to the following: Figure 13 , Figure 13 This is a schematic diagram of the structure of a control device for a robot storage bin provided in an embodiment of this application;

[0234] The control device for the robot storage compartment is applied to the robot; specifically, the control device for the robot storage compartment is applied to one or more processors of the robot.

[0235] like Figure 13 As shown, the control device 130 of the robot storage compartment includes:

[0236] Navigation unit 131 is used to control the robot to move to a designated position;

[0237] The acquisition unit 132 is used to acquire the user's clothing information and determine whether the user has the operation permission to operate the storage container based on the user's clothing information;

[0238] The control unit 133 is used to acquire the user's gesture information if the user has the operation permission to operate the storage compartment, and control the storage compartment according to the operation intention indicated by the gesture information.

[0239] In this embodiment, the control device for the robot storage compartment can also be constructed from hardware components. For example, the control device for the robot storage compartment can be constructed from one or more chips, which can work in coordination to complete the control method for the robot storage compartment described in the above embodiments. As another example, the control device for the robot storage compartment can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0240] The control device for the robotic storage compartment in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0241] The control device for the robotic storage compartment in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not impose specific limitations.

[0242] The control device for the robotic storage bin provided in this application embodiment can achieve... Figure 2To avoid repetition, the various processes involved will not be detailed here.

[0243] It should be noted that the control device for the robot storage compartment described above can execute the control method for the robot storage compartment provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the embodiments of the robot storage compartment control device can be found in the control method for the robot storage compartment provided in the above embodiments.

[0244] In this embodiment, a control device for a robot storage compartment is provided, comprising: a navigation unit for controlling the robot to move to a designated location; an acquisition unit for acquiring a user's clothing information and determining whether the user has permission to operate the storage compartment based on the clothing information; and a control unit for acquiring the user's gesture information if the user has permission to operate the storage compartment, and controlling the storage compartment according to the operation intention indicated by the gesture information. By acquiring the user's clothing information, determining whether the user has permission to operate the storage compartment based on the clothing information, and acquiring the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information, this application can achieve contactless control of the robot's storage compartment.

[0245] Please refer to the following: Figure 14 , Figure 14 This is a schematic diagram of the structure of a robot provided in an embodiment of this application;

[0246] like Figure 14 As shown, the robot 140 includes one or more processors 141 and a memory 142. Wherein, Figure 14 Take a processor 141 as an example.

[0247] Processor 141 and memory 142 can be connected via a bus or other means. Figure 14 Taking the example of a connection between China and Israel via a bus.

[0248] The processor 141 is configured to provide computing and control capabilities to control the robot 140 to perform corresponding tasks, such as controlling the robot 140 to perform the control method of the robot storage compartment in any of the above method embodiments, including controlling the robot to move to a designated position; obtaining the user's clothing information and determining whether the user has the operation permission to operate the storage compartment based on the user's clothing information; if the user has the operation permission to operate the storage compartment, obtaining the user's gesture information and controlling the storage compartment according to the operation intention indicated by the gesture information.

[0249] By acquiring the user's clothing information, determining whether the user has the right to operate the storage compartment based on the clothing information, and if the user has the right to operate the storage compartment, acquiring the user's gesture information, and controlling the storage compartment according to the operation intention indicated by the gesture information, this application can realize contactless control of the robot's storage compartment.

[0250] Processor 141 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0251] Memory 142, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the robot storage bin control method in the embodiments of this application. Processor 141 can implement the robot storage bin control method in any of the following method embodiments by running the non-transitory software programs, instructions, and modules stored in memory 142. Specifically, memory 142 may include volatile memory (VM), such as random access memory (RAM); memory 142 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 142 may also include combinations of the above types of memory.

[0252] Memory 142 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 142 may optionally include memory remotely located relative to processor 141, and such remote memory may be connected to processor 141 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0253] One or more modules are stored in memory 142. When executed by one or more processors 141, they perform the control method of the robot storage compartment in any of the above method embodiments, for example, the method described above. Figure 2 The steps shown can also be implemented. Figure 13 The functions of each module or unit.

[0254] In this embodiment, the robot 140 may also have wired or wireless network interfaces, keyboards, and input / output interfaces for input and output. The robot 140 may also include other components for implementing device functions, which will not be described in detail here.

[0255] The robot in this embodiment exists in the form of a mobile robot, performing the above-described... Figure 2 The steps shown can also be implemented. Figure 13 The functions of each unit include, but are not limited to: delivery robots, transport robots, warehousing robots, and other robots equipped with storage compartments.

[0256] This application also provides a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the control method of the robot storage compartment in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0257] This application also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. The processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the steps of the robot storage bin control method provided in the above embodiments.

[0258] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0259] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0260] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations as described above in different aspects of this application, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A control method of a robotic storage compartment, characterized in that, The robot comprises a storage bin and a camera device, preset verification information comprises first information and second information, wherein the first information is clothing information, and the second information is scene environment information; the method comprises: informing a target user to upload clothing information for personal identity verification and scene environment information for scene verification to the robot through pictures and / or videos; when receiving the clothing information uploaded by the target user, recording the received clothing information as the first information in the preset verification information; when receiving the scene environment information uploaded by the target user, recording the scene environment information as the second information in the preset verification information; controlling the robot to move to a specified position; obtaining clothing information of a user, and determining whether the user has an operation permission of operating the storage bin according to the clothing information of the user; if the user has the operation permission of operating the storage bin, obtaining gesture information of the user, and controlling the storage bin according to an operation intention indicated by the gesture information; the obtaining of the clothing information of the user and the determining of whether the user has the operation permission of operating the storage bin according to the clothing information of the user comprise: collecting scene environment information of a current position of the robot through the camera device arranged on the robot; when the scene environment information of the current position of the robot matches the scene environment information in the preset verification information, obtaining clothing information of a user; when the clothing information of the user matches the clothing information in the preset verification information, determining that the user has the operation permission of operating the storage bin.

2. The method of claim 1, wherein, the obtaining of the clothing information of the user and the determining of whether the user has the operation permission of operating the storage bin according to the clothing information of the user further comprise: when the clothing information of the user does not match the clothing information in the preset verification information, determining that the user does not have the operation permission of operating the storage bin.

3. The method of claim 1, wherein, the clothing information comprises clothing color information and clothing style information; the obtaining of the clothing information of the user and the determining of whether the user has the operation permission of operating the storage bin according to the clothing information of the user further comprise: collecting the clothing color information and the clothing style information of the user through the camera device arranged on the robot; when the clothing color information and the clothing style information of the user both match the clothing information in the preset verification information, determining that the user has the operation permission of operating the storage bin; otherwise, the user does not have the operation permission of operating the storage bin.

4. The method of claim 1, wherein, the obtaining of the gesture information of the user and the controlling of the storage bin according to the operation intention indicated by the gesture information comprise: collecting gesture information of a user who has the operation permission of operating the storage bin through the camera device arranged on the robot; determining a corresponding operation intention according to the gesture information of the user, wherein the operation intention comprises opening a bin door of the storage bin and closing the bin door of the storage bin; when the bin door of the storage bin is in a closed state and the operation intention is to open the bin door of the storage bin, controlling to open the bin door of the storage bin; When the door of the storage bin is in an open state and the operation intention is to close the door of the storage bin, the door of the storage bin is controlled to be closed.

5. The method of claim 1, wherein, If the user has the operation permission of the storage bin, gesture information of the user is acquired, and after the operation intention indicated by the gesture information is used to control the storage bin, the method further includes: acquiring gesture information of the user; controlling the robot to perform a corresponding operation action according to the operation intention of the robot indicated by the gesture information.

6. The method of claim 1, wherein, acquiring clothing information of the user, and determining whether the user has the operation permission of the storage bin according to the clothing information of the user, and the method further includes: acquiring clothing information of the user; when the clothing information of the user does not match the clothing information in the preset verification information, and the preset verification information is a null value, determining that the user has the operation permission of the storage bin.

7. A robot, characterized in that including: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control method of the robot storage bin according to any one of claims 1-6.

Images