Robot entry and exit methods, electronic devices and storage media

Abstract

This application relates to the field of robotics and discloses a method for a robot to enter and exit an elevator, comprising: acquiring the robot's positioning information; determining a first pose relationship between the robot's current position and the elevator entrance based on the positioning information; acquiring a target image of the robot within a preset range of its current position, the target image including a depth image and a visible light image; determining a second pose relationship between the robot's current position and the elevator entrance based on the depth image and the visible light image; and controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship. By controlling the robot to pass through the elevator entrance based on the first pose relationship determined by the positioning information and the second pose relationship determined by the depth image and the visible light image, this application can improve the success rate of the robot entering the elevator.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to a method for a robot to enter and exit an elevator, an electronic device, and a storage medium. Background Technology

[0002] Robots are a common term for automated machines, encompassing all machines that mimic human behavior or thought, or those that imitate other living beings. In related technologies, when a robot rides an elevator, lidar is typically used to determine its relative position to the elevator entrance, thus determining whether the robot has successfully navigated the elevator. However, the high reflectivity of the metal surface of the elevator car can lead to sparse point cloud data received by the lidar, or significant distortion, resulting in large positioning errors. This makes it difficult to accurately determine whether the robot has successfully navigated the elevator, reducing the robot's success rate in riding elevators. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a method for a robot to enter and exit an elevator, an electronic device, and a storage medium to improve the success rate of a robot riding an elevator.

[0004] The embodiments of this application provide the following technical solutions:

[0005] In a first aspect, embodiments of this application provide a method for a robot to enter and exit an elevator, including:

[0006] Obtain the robot's positioning information and determine the robot's current position relative to the elevator entrance based on the positioning information;

[0007] Acquire a target image within a preset range of the robot's current position. The target image includes a depth image and a visible light image. Determine the second pose relationship of the robot's current position relative to the elevator entrance based on the depth image and the visible light image.

[0008] The robot is controlled to pass through the elevator entrance based on the first pose relationship and the second pose relationship.

[0009] In some embodiments, the step of controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship includes:

[0010] Robot-based LiDAR acquires point cloud data, including that of elevator entrances;

[0011] The third pose of the robot relative to the elevator entrance is determined based on point cloud data;

[0012] The robot is controlled to pass through the elevator entrance based on the first pose relationship, the second pose relationship, and the third pose relationship.

[0013] In some embodiments, the step of controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship includes:

[0014] When the robot passes through the elevator entrance, acquire the robot's odometer data and monitor the robot's driving behavior at the elevator entrance based on the odometer data;

[0015] The robot's movement determines whether it has successfully passed through the elevator entrance.

[0016] In some embodiments, the step of monitoring the robot's movement through elevator entrances based on odometer data includes:

[0017] If the door gap feature of the elevator entrance disappears in the target image within a preset range of the robot's current position during the robot's passage through the elevator entrance, and the odometer data continues to increase, then it is determined that the robot is performing the action of passing through the elevator entrance.

[0018] When the robot is facing the elevator entrance, control the robot to travel a preset distance in a straight line according to the odometer data until it successfully passes through the elevator entrance.

[0019] In some embodiments, the step of acquiring the robot's positioning information and determining the first pose relationship between the robot's current position and the elevator entrance based on the positioning information includes:

[0020] A robot-based scanning device acquires an identification code at a pre-set location at the elevator entrance.

[0021] The robot's location information is determined based on the location information of the identification code;

[0022] The robot's current position relative to the elevator entrance is determined based on the positioning information.

[0023] In some embodiments, a scanning device is provided on the top of the robot, and an identification code is provided at a preset position on the top of the elevator car;

[0024] The steps for obtaining an identification code at a pre-set location associated with the elevator entrance using a robot-based scanning device include:

[0025] When the robot is inside the elevator car and needs to leave the elevator car, the robot's scanning device scans the identification code set at a preset position on the top of the elevator car. The associated position of the elevator entrance is the preset position on the top of the elevator car.

[0026] In some embodiments, the step of acquiring the robot's positioning information and determining the first pose relationship between the robot's current position and the elevator entrance based on the positioning information includes:

[0027] When the robot is outside the elevator car and needs to enter the elevator car, the positioning information of the robot is obtained based on the positioning algorithm, and the first pose relationship of the robot's current position relative to the elevator entrance is determined based on the positioning information.

[0028] In some embodiments, the step of determining the second pose relationship between the robot's current position and the elevator entrance based on the depth image and the visible light image includes:

[0029] Based on a pre-trained image recognition model, using depth images and visible light images as input, the elevator entrance is identified;

[0030] The robot's current position relative to the elevator entrance is determined based on the recognition results at the elevator entrance.

[0031] Secondly, embodiments of this application provide an electronic device, including:

[0032] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a method for a robot to enter or exit an elevator, as described in the first aspect.

[0033] Thirdly, embodiments of this application provide a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the robot entering and exiting an elevator method as described in the first aspect.

[0034] The beneficial effects of the embodiments of this application are as follows: Unlike existing technologies, the embodiments of this application provide a method for a robot to enter and exit an elevator, comprising: acquiring the robot's positioning information; determining a first pose relationship between the robot's current position and the elevator entrance based on the positioning information; acquiring a target image of the robot within a preset range of its current position, the target image including a depth image and a visible light image; determining a second pose relationship between the robot's current position and the elevator entrance based on the depth image and the visible light image; and controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship. By controlling the robot to pass through the elevator entrance based on the first pose relationship determined by the positioning information and the second pose relationship determined by the depth image and the visible light image, this application can improve the success rate of the robot taking the elevator. Attached Figure Description

[0035] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0036] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0037] Figure 2 This is a flowchart illustrating a method for a robot to enter and exit an elevator, as provided in an embodiment of this application.

[0038] Figure 3 This is a schematic diagram of the structure of a robot entering and exiting an elevator provided in an embodiment of this application;

[0039] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application.

[0040] Explanation of icon numbers:

[0041] Label name Label name 100 Application Environment 302 Second pose determination unit 10 robot 303 Control Unit 20 elevator 400 electronic devices 300 Robot entering and exiting elevator device 401 processor 301 First pose determination unit 402 memory Detailed Implementation

[0042] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as "connected to" another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this specification are for illustrative purposes only.

[0043] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0044] The technical solution of this application is described in detail below with reference to the accompanying drawings:

[0045] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application.

[0046] like Figure 1 As shown, the application environment 100 includes: robot 10 and elevator 20.

[0047] In this embodiment, the robot 10 includes mobile robots, such as hotel robots, delivery robots, cleaning robots, pet robots, handling robots, care robots, remote monitoring robots, sweeping robots, and other robots.

[0048] The robot includes a main body, drive wheels, a camera, sensors, and a controller. The main body can be generally elliptical, triangular, D-shaped, or other shapes. The controller is located on the main body, and the drive wheels are mounted on the main body to drive the robot's movement.

[0049] In this embodiment, the drive wheel component includes a left drive wheel, a right drive wheel, and an omnidirectional wheel. The left and right drive wheels are respectively mounted on opposite sides of the main body. The omnidirectional wheel is mounted at the front of the bottom of the main body and is a movable caster wheel that can rotate 360 ​​degrees horizontally, allowing the robot to turn flexibly. The mounting of the left drive wheel, right drive wheel, and omnidirectional wheel forms a triangle to improve the stability of the robot's movement.

[0050] In the embodiments of this application, the camera includes, but is not limited to, infrared cameras, night vision cameras, network cameras, digital cameras, high-definition cameras, 4K cameras, 8K high-definition cameras and other camera devices; the sensor includes, but is not limited to, LiDAR, depth cameras, visible light cameras, RGBD cameras and odometers.

[0051] In this embodiment, the controller is located inside the main body and is electrically connected to the left drive wheel, right drive wheel, and omnidirectional wheel. As the robot's control core, the controller is used to control the robot's passage through elevator entrances and perform some business logic processing. For example, the controller is used to plan a global static path. The controller uses Simultaneous Localization and Mapping (SLAM) technology, i.e., employing the SLAM algorithm for positioning and navigation, or integrates a positioning module and an Inertial Measurement Unit (IMU) to achieve positioning and navigation. The positioning module can be a Global Positioning System (GPS) positioning module, a BeiDou Navigation Satellite System (BDS) positioning module, or a positioning module based on other positioning systems and / or positioning algorithms; this application does not impose any limitations on this.

[0052] In this embodiment, the elevator 20 includes an electrically driven car elevator, such as a passenger elevator, a freight elevator, a service elevator, etc. The elevator 20 stops at at least two floors so that the robot can ride to different floors.

[0053] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for a robot to enter and exit an elevator, as provided in an embodiment of this application.

[0054] In a first aspect, embodiments of this application propose a method for a robot to enter and exit an elevator. This method is applied to an electronic device, such as a robot. Specifically, the execution entity of this method is one or at least two processors of the robot.

[0055] like Figure 2 As shown, the method for the robot to enter and exit the elevator includes steps S201-S203:

[0056] Step S201: Obtain the robot's positioning information and determine the robot's current position relative to the elevator entrance based on the positioning information.

[0057] Specifically, the robot's localization information includes the robot's position and orientation (direction), the elevator entrance includes the elevator entrance, and the first-position relationship includes the robot's current position and orientation (direction) relative to the elevator entrance.

[0058] For example, by using a positioning algorithm or obtaining location markers calibrated on-site, the robot's positioning information in the map can be determined, and the pose of the elevator entrance in the map can be obtained, thereby determining the robot's first pose relative to the elevator entrance based on the robot's positioning information and the pose of the elevator entrance.

[0059] In some embodiments, the step of obtaining robot positioning information and determining the first pose relationship of the robot's current position relative to the elevator entrance based on the positioning information includes: obtaining an identification code preset at an associated location of the elevator entrance based on the robot's scanning device; determining the robot's positioning information based on the location information of the identification code; and determining the first pose relationship of the robot's current position relative to the elevator entrance based on the positioning information.

[0060] The robot is equipped with a scanning device on its top, such as a camera or barcode scanner. This device scans and identifies identification codes, such as QR codes or barcodes. Optionally, the associated location of the elevator entrance includes a preset location on the top of the elevator car and / or a preset location outside the elevator. Each elevator entrance may have one or more associated locations, all located near the elevator entrance. For example, the associated location of the elevator entrance may be a preset location on the top of the elevator car, optionally at the center of the top of the elevator car. Another example is a preset location outside the elevator car, optionally on the wall next to the elevator door. The identification code is pre-installed at the associated location of the elevator entrance. When the scanning device scans and identifies the identification code, it acquires the location information of the identification code, which is the location information of the identification code on a map. In this embodiment, the location information of the identification code is stored in the QR code information corresponding to the identification code. The step of obtaining an identification code at a preset location associated with the elevator entrance using a robot-based scanning device includes: when the robot is inside the elevator car and needs to leave the elevator car, the robot-based scanning device scans the identification code set at a preset location on the top of the elevator car; or, when the robot is outside the elevator car, the robot-based scanning device scans the identification code set at a preset location outside the elevator car.

[0061] For example, when a robot is inside an elevator car and needs to leave, its processor controls a camera mounted on top of the robot to scan a QR code located in the center of the elevator car's top. The camera then analyzes the QR code to obtain its location information, i.e., its position on a map. Based on the image of the QR code captured by the camera, the robot's relative pose to the QR code is determined. Through coordinate system transformation, the robot's position and orientation on the map are then determined.

[0062] In this embodiment, identification codes (QR codes) are pre-marked at preset locations on the map of each working floor. When the robot scans and recognizes the QR code, its relative pose with respect to the QR code is determined. Thus, the robot's current position relative to the elevator entrance is determined, i.e., its position and orientation (direction). The robot stores maps of each floor, and pre-saved markers on the maps are used to indicate the location of the elevator entrance.

[0063] For example, when the robot is outside the elevator car, the robot's processor controls a camera mounted on top of the robot to scan an identification code at a preset location outside the elevator car. The camera then parses the code to obtain its location information on a map of the current floor. This location information includes the identification code's position and orientation on the map. In other embodiments, the identification code also carries the floor number information. When the robot scans and identifies the identification code, it parses and obtains the floor number information, loads the map corresponding to the current floor number, and performs positioning and navigation. In other embodiments, the step of obtaining the robot's positioning information and determining the robot's first-position orientation relative to the elevator entrance based on the positioning information includes: when the robot is outside the elevator car and needs to enter the elevator car, obtaining the robot's positioning information based on a positioning algorithm, and determining the robot's first-position orientation relative to the elevator entrance based on the positioning information.

[0064] For example, localization algorithms estimate robot pose based on acquired laser point clouds and / or visual point clouds and pre-built SLAM maps to obtain robot localization information. Of course, when navigating a robot, more sensor data can be fused to estimate the robot's pose and determine its localization information.

[0065] Specifically, the robot, including its LiDAR, determines its position and orientation on the current floor's map by matching the acquired LiDAR point cloud with the map of the floor it's on, combined with data from sensors such as odometry or IMU. Then, it uses pre-saved markers on the map to determine the elevator entrance's location, thus determining the robot's current position and orientation relative to the elevator entrance. It's understood that the map of the floor the robot is on has already been pre-built by the robot during runtime using Simultaneous Localization and Mapping (SLAM) technology, i.e., the SLAM algorithm.

[0066] Step S202: Obtain a target image of the robot within a preset range at its current position. The target image includes a depth image and a visible light image. Determine the second pose relationship of the robot's current position relative to the elevator entrance based on the depth image and the visible light image.

[0067] Specifically, the target image includes a depth image and a visible light image of the elevator entrance, and the second pose relationship includes the robot's current position and orientation (direction) relative to the elevator entrance. For example, depth and visible light images within the camera's field of view are acquired using a depth camera and a visible light camera, or an RGBD camera, and then the second pose relationship of the robot's current position relative to the elevator entrance is determined based on the depth and visible light images and a pre-trained image recognition model.

[0068] In some embodiments, the robot includes a depth camera and a visible light camera, which are mounted as a single module at an angle downwards at the front end of the robot. The depth camera is used to acquire depth images, and the visible light camera is used to acquire visible light images. The preset range includes the field of view of the depth camera and the visible light camera, i.e., the area that the camera can capture.

[0069] In other embodiments, the robot also includes an RGBD camera mounted at an angle downwards at the front end of the robot. The RGBD camera is used to acquire depth images and visible light images. The preset range includes the field of view of the RGBD camera, that is, the area that the camera can capture.

[0070] The step of determining the second pose relationship between the robot's current position and the elevator entrance based on the depth image and the visible light image includes: identifying the elevator entrance based on a pre-trained image recognition model, using the depth image and the visible light image as input; and determining the second pose relationship between the robot's current position and the elevator entrance based on the identification result of the elevator entrance.

[0071] The image recognition model is used to identify elevator entrances. The identification result of the elevator entrance includes the identified elevator entrance features. Optionally, the elevator entrance features include the door gap features of the elevator entrance. In some embodiments, the training process of the image recognition model includes: (1) acquiring an image dataset, wherein the image dataset includes a depth image and a visible light image of the elevator entrance. (2) training the image recognition model through deep learning based on the image dataset to obtain the trained image recognition model. The image recognition model detects the door gap features of the elevator entrance in the visible light image and matches them with the depth image corresponding to the visible light image that detects the door gap features of the elevator entrance using pre-calibrated parameters. When the depth image matches the depth features, the elevator in the visible light image is identified as an elevator entrance. The depth features are characterized by most points in the area being located on the ground and a small portion being located below the ground. Since the visible light image features of the elevator door gap are mainly black and gray long rectangles, which are easily confused with ordinary stripes, common negative samples, such as dark stripes on clothing, should be added when training the image recognition model.

[0072] After the image recognition model is trained, it is used to detect the input depth and visible light images to identify the elevator entrance. Once the elevator entrance is identified, the position of the elevator door gap center in the image is substituted into a mapping relationship established by pre-calibrated camera intrinsic and extrinsic parameters to calculate the robot's current position and orientation relative to the elevator entrance. The camera intrinsic parameters describe the transformation from the camera coordinate system to the image pixel coordinate system and are related to the camera structure. The camera extrinsic parameters describe the transformation from the world coordinate system to the camera coordinate system and are related to the camera's installation position. Combining these two parameters establishes a mapping relationship between image pixels and spatial coordinates in the robot's coordinate system. The depth and visible light cameras can be calibrated separately. If both are in the same module and high accuracy is not required, only the extrinsic parameters of one camera can be calibrated, and the extrinsic parameters of the other camera can be obtained through hardware structural parameter conversion. It is understandable that since the target is the elevator door gap, which can be considered as being on the ground, and the depth image contains depth information, combining multiple frames of data can yield the robot's current pose relative to the elevator entrance, i.e., position and orientation (orientation).

[0073] In this embodiment of the application, the elevator entrance is identified by using a pre-trained image recognition model with depth image and visible light image as input. This application can eliminate misidentified targets far from the ground by using depth image, and avoid the situation where other horizontal strip targets are easily misidentified as elevator door gaps when detecting elevator door gaps based on visible light image.

[0074] Step S203: Control the robot to pass through the elevator entrance according to the first pose relationship and the second pose relationship.

[0075] In this embodiment, by determining the first pose and second pose of the robot relative to the elevator entrance, multiple sensor data, such as laser point cloud data, visual point cloud data, depth images, and visible light images, can be fused to achieve robot positioning. When the robot passes through the elevator entrance, it can smoothly enter and exit the elevator, avoid bumping into the elevator entrance wall, or avoid situations where entering and exiting the elevator is inefficient.

[0076] Specifically, when an identification code is set at a preset position on the top of the elevator car and / or at a preset position outside the elevator car, the robot is controlled to pass through the elevator entrance according to the first posture relationship; when no identification code is set at a preset position on the top of the elevator car and no identification code is set at a preset position outside the elevator car, the robot is controlled to pass through the elevator entrance according to the second posture relationship.

[0077] In some embodiments, the step of controlling the robot to pass through an elevator entrance based on a first pose relationship and a second pose relationship includes: acquiring point cloud data including the elevator entrance based on the robot's LiDAR; determining a third pose relationship of the robot relative to the elevator entrance based on the point cloud data; and controlling the robot to pass through the elevator entrance based on the first pose relationship, the second pose relationship, and the third pose relationship.

[0078] Specifically, the third pose relationship includes the robot's position and orientation relative to the elevator door. When the robot faces the elevator door, it acquires point cloud data including the elevator door using LiDAR. Based on the geometric features of the elevator door, such as symmetry, and the discontinuous nature of the point cloud in front of the robot when the door is open, the state of the elevator door and the robot's third pose relationship relative to the elevator door are determined. For example, if the received point cloud data is continuous and symmetrical, the elevator door is determined to be closed; if the received point cloud data is discontinuous with gaps, the elevator door is determined to be open, and the midpoint of the gap in the received point cloud data is the door seam.

[0079] The elevator entrance is represented by its midpoint. When the elevator door is closed, the robot uses a localization algorithm to calculate its position and orientation relative to the elevator door based on the spatial information contained in the point cloud data, thus determining the robot's pose relative to the elevator entrance. When the elevator door is open, the robot uses a localization algorithm to calculate its position relative to the two side walls of the elevator entrance, and determines the robot's pose relative to the elevator entrance.

[0080] Furthermore, controlling the robot's passage through the elevator entrance based on the first pose relationship, the second pose relationship, and the third pose relationship includes: when an identification code is set at a preset position on the top of the elevator car, controlling the robot's passage through the elevator entrance based on the first pose relationship; and in the point cloud data acquired by the LiDAR, there are few dynamic obstacles (usually pedestrians) and they are relatively close to the ground. Figure 1 When the alignment is good, and the elevator door is significantly narrower than the car and the external environment, the robot is controlled to pass through the elevator entrance based on the third pose relationship. In other cases, the robot is controlled to pass through the elevator entrance based on the second pose relationship.

[0081] In the embodiments of this application, by controlling the robot to pass through the elevator entrance according to different pose relationships in different scenarios, this application can improve the success rate of the robot in taking elevator tasks in complex and ever-changing environments.

[0082] In other embodiments, the step of controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship includes: acquiring the robot's odometer data when the robot passes through the elevator entrance, and monitoring the robot's driving behavior at the elevator entrance based on the odometer data; and determining whether the robot has successfully passed through the elevator entrance based on the robot's driving behavior.

[0083] The steps for monitoring the robot's movement through the elevator entrance based on odometer data include: If, during the robot's passage through the elevator entrance, the door gap feature of the elevator entrance disappears in the target image within a preset range of the robot's current position, and the odometer data continues to increase, then it is determined that the robot is performing the action of passing through the elevator entrance; when the robot is facing the elevator entrance, it is controlled to travel a preset distance in a straight line according to the odometer data until it successfully passes through the elevator entrance. The preset range includes the field of view of a depth camera and a visible light camera, or the field of view of an RGBD camera, and the preset distance is greater than the distance between the robot's current position and the elevator entrance.

[0084] Furthermore, if the robot's driving behavior is that it is performing the action of passing through the elevator entrance and travels a preset distance in a straight line when it is facing the elevator entrance, then it is determined that the robot has successfully passed through the elevator entrance.

[0085] Understandably, robot localization primarily relies on LiDAR and odometers. However, the presence of other people in the elevator, or the high reflectivity of the elevator car's interior walls, can interfere with LiDAR. Furthermore, when the robot is inside the elevator with the doors closed, it's difficult to find fixed and easily identifiable reference points, making it unsuitable for using cameras to detect changes in direction. Therefore, for small-scale, localized movements, odometer data is more reliable. Monitoring the robot's movement through elevator entrances based on odometer data allows for a more accurate determination of whether the robot has successfully navigated the elevator.

[0086] In other embodiments, if the door gap feature of the elevator entrance disappears in the target image of the robot within a preset range at the current position, the odometer data is missing, and there are no obstacles within the required movement range, the robot is controlled to travel a preset distance in a straight line until it successfully passes through the elevator entrance.

[0087] For example, if a hardware connection problem with the odometer leads to data loss, to prevent the robot from getting stuck in the elevator door and affecting other passengers, the robot, during its passage through the elevator entrance, uses an image recognition model to identify the disappearance of the door gap feature in the target image, and the robot detects no obstacles ahead using LiDAR (manifested as no obstacle points in the point cloud in front of the robot at close range). Furthermore, the depth image from the depth camera does not detect any obstacles above or below the LiDAR height within the required movement range, nor does it detect any falling environments such as steps or steep slopes (including elevator malfunctions and floor misalignments). In this case, a positioning algorithm controls the robot to travel a preset distance in a straight line until it successfully passes through the elevator entrance. The preset range includes the field of view of the depth camera and visible light camera, or the field of view of the RGBD camera. The required movement range includes the area between the robot's current position and the elevator entrance, and the preset distance is greater than the distance between the robot's current position and the elevator entrance.

[0088] In other embodiments, if the robot is initially positioned with its back to the elevator entrance, and if the odometer data determines that the robot has not changed direction, and if the robot's door gap features are acquired in a target image within a preset range at its current position during its movement, then the robot is deemed to have successfully passed through the elevator entrance. The preset range includes the field of view of a depth camera and a visible light camera, or the field of view of an RGBD camera.

[0089] For example, when the robot is initially facing away from the elevator entrance, the images obtained by the depth camera and visible light camera do not show the elevator entrance. However, if the robot is pushed and its wheels slip, and it starts moving, the odometer will detect that the robot has not changed direction. Furthermore, if the robot recognizes the door gap feature of the elevator entrance in the target image through the image recognition model during its movement, it will be determined that the robot has successfully passed through the elevator entrance while facing away from it.

[0090] Please see Figure 3 , Figure 3 This is a schematic diagram of a robot entering and exiting an elevator, provided in an embodiment of this application.

[0091] Secondly, this application provides a robot elevator entry and exit device, which is applied to electronic devices, such as robots. Specifically, the robot elevator entry and exit device is configured on a robot.

[0092] like Figure 3 As shown, the robot's entry and exit elevator device 300 includes:

[0093] The first pose determination unit 301 is used to acquire the robot's positioning information and determine the first pose relationship between the robot's current position and the elevator entrance based on the positioning information.

[0094] The second pose determination unit 302 is used to acquire a target image of the robot within a preset range at its current position. The target image includes a depth image and a visible light image. Based on the depth image and the visible light image, the second pose relationship of the robot's current position relative to the elevator entrance is determined.

[0095] Control unit 303 is used to control the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship.

[0096] In some embodiments of this application, the first pose determination unit 301 is further configured to obtain an identification code of a preset associated position at the elevator entrance based on the robot's scanning device; determine the robot's positioning information based on the position information of the identification code; and determine the first pose relationship between the robot's current position and the elevator entrance based on the positioning information.

[0097] In some embodiments of this application, a scanning device is provided on the top of the robot; the associated position of the elevator entrance includes a preset position on the top of the elevator car and / or a preset position outside the elevator; the first pose determination unit 301 is further configured to scan the identification code set at the preset position on the top of the elevator car based on the robot's scanning device when the robot is inside the elevator car and needs to leave the elevator car; or, when the robot is outside the elevator car, scan the identification code set at the preset position outside the elevator car based on the robot's scanning device.

[0098] In some embodiments of this application, the first pose determination unit 301 is further configured to obtain the robot's positioning information based on a positioning algorithm when the robot is located outside the elevator car and needs to enter the elevator car, and determine the first pose relationship between the robot's current position and the elevator entrance based on the positioning information.

[0099] In some embodiments of this application, the second pose determination unit 302 is further configured to identify the elevator entrance based on a pre-trained image recognition model, using a depth image and a visible light image as input; and to determine the second pose relationship between the robot's current position and the elevator entrance based on the identification result of the elevator entrance.

[0100] In some embodiments of this application, the control unit 303 is also used to acquire point cloud data including the elevator entrance using the robot's lidar; determine a third pose relationship of the robot relative to the elevator entrance based on the point cloud data; and control the robot to pass through the elevator entrance based on the first pose relationship, the second pose relationship, and the third pose relationship.

[0101] In some embodiments of this application, the control unit 303 is also used to acquire the robot's odometer data when the robot passes through the elevator entrance, and monitor the robot's driving behavior at the elevator entrance based on the odometer data; and determine whether the robot has successfully passed through the elevator entrance based on the robot's driving behavior.

[0102] In some embodiments of this application, the control unit 303 is also used to determine that the robot is performing the action of passing through the elevator entrance if the door gap feature of the elevator entrance disappears in the target image of the robot within a preset range at the current position and the odometer data continues to increase during the process of the robot passing through the elevator entrance; when the robot is facing the elevator entrance, control the robot to travel a preset distance in a straight line according to the odometer data until it successfully passes through the elevator entrance.

[0103] Understandably, the implementation principle and technical effect of the robot entering and exiting the elevator device 300 proposed in the second aspect of this application can be found in the implementation principle and technical effect of the robot entering and exiting the elevator method proposed in the first aspect, and will not be repeated here.

[0104] Please refer to the following: Figure 4 , Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application.

[0105] like Figure 4 As shown, the electronic device 400 includes one or more processors 401 and a memory 402. The electronic device 400 can be a mobile robot, an unmanned vehicle, etc. Figure 4 Take a processor 401 as an example.

[0106] The processor 401 and the memory 402 can be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0107] The processor 401 is configured to provide computing and control capabilities to control the electronic device 400 to perform corresponding tasks, such as controlling the electronic device 400 to perform the robot entering and exiting an elevator method in any of the above method embodiments. The robot entering and exiting an elevator method includes: acquiring the robot's positioning information; determining the first pose relationship between the robot's current position and the elevator entrance based on the positioning information; acquiring a target image of the robot within a preset range of its current position, the target image including a depth image and a visible light image; determining the second pose relationship between the robot's current position and the elevator entrance based on the depth image and the visible light image; and controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship.

[0108] By controlling the robot to pass through the elevator entrance based on the first pose relationship determined by the positioning information and the second pose relationship determined by the depth image and the visible light image, this application can improve the success rate of the robot taking the elevator.

[0109] Processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0110] Memory 402, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the robot entering and exiting the elevator method in the embodiments of this application. Processor 401 can implement the robot entering and exiting the elevator method in any of the above method embodiments by running the non-transitory software programs, instructions, and modules stored in memory 402. Specifically, memory 402 may include volatile memory (VM), such as random access memory (RAM); memory 402 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 402 may also include combinations of the above types of memory.

[0111] Memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, which can be connected to processor 401 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] One or more modules are stored in memory 402. When executed by one or more processors 401, they perform the robot entering and exiting the elevator method in any of the above method embodiments, for example, performing the above-described... Figure 2 The steps shown can also be implemented. Figure 3 The functions of each module or unit.

[0113] In this embodiment, the electronic device 400 may also have wired or wireless network interfaces, keyboards, and input / output interfaces for input and output. The electronic device 400 may also include other components for implementing device functions, which will not be described in detail here.

[0114] The robot in this application embodiment exists in various forms, performing the above-described... Figure 2 The steps shown can also be implemented. Figure 3 The functions of each unit include, but are not limited to: hotel robots, delivery robots, cleaning robots, service robots, remote monitoring robots, sweeping robots, and other robots.

[0115] This application also provides a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the robot entering and exiting an elevator method described in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0116] This application also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. The processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the method steps of the robot entering and exiting an elevator provided in the above embodiments.

[0117] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations as described above in different aspects of this application, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for a robot to enter and exit an elevator, characterized in that, The method includes: Obtain the robot's positioning information, and determine the robot's current position relative to the elevator entrance based on the positioning information; Acquire a target image of the robot within a preset range at its current position. The target image includes a depth image and a visible light image. Determine the second pose relationship of the robot's current position relative to the elevator entrance based on the depth image and the visible light image. The robot is controlled to pass through the elevator entrance based on the first pose relationship and the second pose relationship; The steps of controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship include: When the robot passes through the elevator entrance, the odometer data of the robot is acquired, and the driving behavior of the robot passing through the elevator entrance is monitored based on the odometer data, including: if the door gap feature of the elevator entrance disappears in the target image of the robot within a preset range at the current position during the robot's passage through the elevator entrance, and the odometer data continues to increase, then it is determined that the robot is performing the action of passing through the elevator entrance.

2. The method according to claim 1, characterized in that, The steps of controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship include: Robot-based lidar acquires point cloud data including the elevator entrance; The third pose of the robot relative to the elevator entrance is determined based on the point cloud data; The robot is controlled to pass through the elevator entrance based on the first pose relationship, the second pose relationship, and the third pose relationship.

3. The method according to claim 1, characterized in that, The step of controlling the robot to pass through the elevator entrance based on the first pose relationship and the second pose relationship further includes: The success of passage through the elevator entrance is determined based on the robot's movement.

4. The method according to claim 1, characterized in that, The step of monitoring the robot's movement through the elevator entrance based on the odometer data also includes: When the robot is facing the elevator entrance, the robot is controlled to travel a preset distance in a straight line according to the odometer data until it successfully passes through the elevator entrance.

5. The method according to claim 1, wherein The steps of acquiring the robot's positioning information and determining the robot's current position relative to the elevator entrance based on the positioning information include: A robot-based scanning device acquires an identification code preset at an associated location at the elevator entrance; The robot's location information is determined based on the location information of the identification code; The robot's current position relative to the elevator entrance is determined based on the positioning information.

6. The method according to claim 5, characterized in that, The robot is equipped with a scanning device on its top; the identification code is set at a preset position on the top of the elevator car. The step of obtaining an identification code preset at a related location at the elevator entrance using a robot-based scanning device includes: When the robot is inside the elevator car and needs to leave the elevator car, the robot's scanning device scans the identification code set at a preset position on the top of the elevator car, wherein the associated position of the elevator entrance is the preset position on the top of the elevator car.

7. The method according to claim 1, characterized in that, The steps of acquiring the robot's positioning information and determining the robot's current position relative to the elevator entrance based on the positioning information include: When the robot is outside the elevator car and needs to enter the elevator car, the robot's positioning information is obtained based on the positioning algorithm, and the first pose relationship between the robot's current position and the elevator entrance is determined based on the positioning information.

8. The method according to claim 1, characterized in that The step of determining the second pose relationship between the robot's current position and the elevator entrance based on the depth image and the visible light image includes: Based on a pre-trained image recognition model, the elevator entrance is identified using the depth image and the visible light image as input. The robot's current position relative to the elevator entrance is determined based on the recognition result of the elevator entrance.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the robot entering and exiting the elevator method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the robot entering and exiting the elevator method as described in any one of claims 1 to 8.

Images