Biped robot task navigation method and related device based on image segmentation

Abstract

This application discloses a bipedal robot task navigation method and related device based on image segmentation. The method includes: obtaining multiple feature objects obtained by performing image segmentation based on panoramic image information collected by a perception module at a target location; determining whether there is a target vacant position within the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot, and the spatial features of the target location; upon detecting the presence of the target vacant position within the target location, determining a target behavior strategy based on the person recognition results of the multiple feature objects; and controlling the bipedal robot to execute the target behavior strategy. By performing multiple detections on the feature objects obtained after image segmentation of the collected panoramic image, the robot's target behavior strategy is determined based on the detection results, and the robot completes the elevator delivery task based on the conditions and self-service decision-making, without the need for assistance from other elevator passengers, which is conducive to improving the intelligence of the robot in performing the elevator delivery task.

Description

Technical Field

[0001] The present application relates to the field of intelligent robots, and in particular to a bipedal robot task navigation method and related devices based on image segmentation. Background Art

[0002] With the continuous development of artificial intelligence and intelligent robotics, robots that ride elevators and deliver items to users on higher floors have become a highly sought-after new development to meet people's growing demand for convenience. However, autonomous elevator use by robots currently faces several challenges. For example, their movements are often clunky, requiring other passengers to actively assist them, resulting in a lack of intelligence when performing such tasks. Summary of the Invention

[0003] The embodiment of the present application provides a bipedal robot task navigation method and related devices based on image segmentation. By performing image segmentation on the collected panoramic image, the identified feature objects are detected multiple times, and finally the robot's target behavior strategy is determined according to the detection results. The robot completes the elevator delivery task based on the conditions and makes self-service decisions without the need for assistance from other elevator passengers. This is conducive to improving the intelligence of the robot in performing the elevator delivery task.

[0004] In a first aspect, an embodiment of the present application provides a bipedal robot task navigation method based on image segmentation, which is applied to a control module of a bipedal robot, wherein the bipedal robot includes the control module and a perception module in communication with the control module. The method includes:

[0005] Acquire multiple feature objects obtained by image segmentation based on panoramic image information collected by the perception module at the target location; determine whether there is a target idle position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot and the spatial features of the target location; detect the existence of the target idle position in the target location, and determine a target behavior strategy based on the character recognition results of the multiple feature objects, the target behavior strategy is used to complete the elevator task navigation of the bipedal robot; and control the bipedal robot to execute the target behavior strategy.

[0006] In a second aspect, an embodiment of the present application provides a bipedal robot task navigation device based on image segmentation, comprising:

[0007] An acquisition module, configured to acquire a plurality of feature objects obtained by performing image segmentation based on the panoramic image information collected by the perception module at the target location;

[0008] A processing module is used to determine whether there is a target idle position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot and the spatial features of the target location; and is used to detect the existence of the target idle position in the target location and determine a target behavior strategy based on the character recognition results of the multiple feature objects, the target behavior strategy is used to complete the elevator task navigation of the bipedal robot; and is used to control the bipedal robot to execute the target behavior strategy.

[0009] In a third aspect, an embodiment of the present application provides a computer, including:

[0010] A memory, a processor, and an executable program code stored in the memory and capable of running on the processor, wherein the processor executes the method as described in any one of the first aspects when executing the executable program code.

[0011] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which is stored a bipedal robot task navigation program based on image segmentation, wherein the bipedal robot task navigation program based on image segmentation includes execution instructions, and the execution instructions are used to execute the method as described in any one of the first aspects.

[0012] In a fifth aspect, the present application provides a computer program product, which, when executed by a processor, is used to implement the method as described in any one of the first aspects.

[0013] By implementing the embodiments of the present application, the control module of the bipedal robot first obtains multiple feature objects obtained by performing image segmentation based on the panoramic image information collected by the perception module at the target location, and then determines whether there is a target free position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot, and the spatial features of the target location. Then, when the target free position is detected in the target location, the target behavior strategy is determined based on the human recognition results of the multiple feature objects. The target behavior strategy is used to complete the bipedal robot's elevator task navigation, and finally the bipedal robot is controlled to execute the target behavior strategy. By performing image segmentation on the collected panoramic image, multiple detections are performed on the identified feature objects, and finally the target behavior strategy of the robot is determined based on the detection results, the elevator delivery task is completed based on the self-service decision-making according to the conditions, without the need for other elevator passengers to provide assistance, which is conducive to improving the intelligence of the robot in performing the elevator delivery task. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0015] Figure 1This is a schematic diagram of the architecture of a bipedal robot provided in an embodiment of the present application;

[0016] Figure 2 This is a flowchart of a biped robot task navigation method based on image segmentation provided in an embodiment of the present application;

[0017] Figure 3 This is a schematic diagram of a scene for image segmentation using the SAM2 model provided in an embodiment of the present application;

[0018] Figure 4 This is a schematic diagram of a scene before a bipedal robot enters an elevator, provided in an embodiment of the present application;

[0019] Figure 5 This is a schematic diagram of a scene after a bipedal robot enters an elevator provided by an embodiment of the present application;

[0020] Figure 6 This is a schematic diagram of a terminal interface of a teleoperation system of a bipedal robot provided in an embodiment of the present application;

[0021] Figure 7 This is a schematic diagram of a mode adjustment interface of a bipedal robot provided in an embodiment of the present application;

[0022] Figure 8 This is a structural diagram of a bipedal robot task navigation device based on image segmentation provided in an embodiment of the present application;

[0023] Figure 9 This is a structural diagram of a computer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] The terms "first," "second," and "third," etc. in the specification, claims, and drawings of this application are used to distinguish between different objects, not to describe a particular order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0026] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0027] See also Figure 1 , Figure 1 This is a schematic diagram of the architecture of a bipedal robot provided in an embodiment of the present application. Figure 1 As shown, the image segmentation-based bipedal robot 100 includes a perception module 101 and a control module 102. The perception module 101 includes sensors (such as cameras, depth cameras, and lidar) for acquiring environmental information and images of the surrounding scene. The perception module 101 also processes sensor data, including image preprocessing, image segmentation, and feature extraction, to identify characteristic objects in the image, including surrounding objects, obstacles, and human users. The control module 102 is in communication with the perception module 101 and includes decision-making and control execution functions. These functions include making decisions and planning paths based on information provided by the perception module 101, thereby determining the robot's behavior and movements. The control module 102 also converts decision outputs into specific actions and controls the specific rotation angles, travel speeds, and travel trajectories of actuators, such as robotic arms, wheeled drives, or other motion devices. The control module 102 issues control signals to enable the robot to move, operate, or interact with the outside world.

[0028] Among them, the bipedal robot 100 is a robot designed and constructed to resemble the structure of human legs. The bipedal robot 100 imitates the gait and movement of humans, and uses two mechanical legs with joints to achieve walking, standing and performing various tasks. Bipedal robots are usually composed of multiple electric joints, sensors and control systems. The multiple electric joints can simulate the degrees of freedom of human joints and achieve balance, walking and other movements through precise control. Sensors are used to measure environmental information, such as visual sensors, depth sensors, tactile sensors and inertial measurement units (IMUs). The control system is responsible for calculating and sending instructions to maintain balance, adjust gait and perform other tasks. The bipedal robot 100 also includes an energy system that provides the robot with the required electricity or other forms of energy. Preferably, the bipedal robot 100 is a humanoid robot.

[0029] Based on this, the present application provides a biped robot task navigation method and related devices based on image segmentation. The present application is described in detail below with reference to the accompanying drawings.

[0030] See also Figure 2 , Figure 2 This is a flowchart of a biped robot task navigation method based on image segmentation provided by an embodiment of the present application, such as Figure 2 As shown, the method includes the following steps:

[0031] S201, obtaining a plurality of feature objects obtained by performing image segmentation on the panoramic image information collected at the target location by the perception module.

[0032] The perception module for collecting panoramic image information of the target location may be one or more devices such as a camera and a depth camera, and is not limited here.

[0033] Panoramic image information refers to all visual data captured within a continuous horizontal coverage area of ​​the entire scene or environment as seen from the robot's position. This data includes elements such as visible objects, background, and terrain in the scene, and can provide a rich and detailed visual reference. Preferably, in this solution, when the robot performs an elevator delivery task, the target location is the elevator, and the panoramic image information collected is all visual data inside the elevator.

[0034] Among them, the method of image segmentation for panoramic image information can be threshold-based image segmentation, region growing algorithm, edge detection algorithm, deep learning, etc., which is not limited here.

[0035] In a possible implementation, obtaining multiple feature objects obtained by performing image segmentation by the perception module based on panoramic image information collected at the target location includes:

[0036] Acquire first panoramic image information collected by the perception module at the target location, wherein the panoramic image information includes human elements, object elements, and background elements; acquire second panoramic image information after the perception module performs preprocessing operations on the first panoramic image information, wherein the preprocessing operations include one or more of denoising, translation and rotation, image size adjustment, and brightness and contrast adjustment; acquire image segmentation of the second panoramic image information by the perception module according to the object segmentation model 2 in the machine learning model to obtain multiple feature objects.

[0037] Among them, human elements refer to human images appearing in the image, including human bodies in different postures and actions such as walking, standing, and sitting, as well as their physical characteristics such as facial expressions and clothing; object elements refer to other actual objects in the image other than humans. These objects can be various daily necessities, furniture, vehicles, buildings, etc., with different shapes, sizes and textures. In this solution, object elements refer to all objects inside the elevator except human elements and background elements; background elements are the parts behind the characters and objects that constitute the background of the entire scene in the image. The background usually includes the sky, ground, walls, etc. In this solution, background elements refer to the elevator walls and elevator floor.

[0038] The preprocessing operation on the first panoramic image information includes one or more of the following steps:

[0039] Image denoising: Use filters (such as Gaussian filters and median filters) to smooth images and reduce noise;

[0040] Image enhancement: adjust image contrast, brightness and other parameters;

[0041] Image scaling and cropping: Scale or crop images as needed to fit specific size requirements or areas of interest.

[0042] Histogram equalization: Enhances the contrast of the entire image by redistributing the gray levels;

[0043] Feature extraction: Use computer vision algorithms (such as edge detection and corner detection) to extract key feature points or boundary information for tasks such as target detection and tracking.

[0044] Among them, Segment Anything Model 2 (SAM2) is an image segmentation model currently used to segment different targets in unstructured frames. SAM2 provides real-time, promptable object segmentation for static images and dynamic video content, unifying image and video segmentation functions into a powerful system. Specifically, SAM2 adopts a streaming memory design and achieves accurate tracking and segmentation of objects in video frames through a combination of memory encoders, memory banks, and memory attention modules. The functions of each component are as follows:

[0045] ‌Memory Encoder‌: Responsible for extracting key information from the input video frames, which is used for subsequent object recognition and segmentation.

[0046] ‌Memory‌: Stores key information and object features learned by the model while processing the video, allowing for fast matching and segmentation in subsequent frames.

[0047] ‌Memory Attention Module‌: Helps the model more accurately localize and segment objects in videos by computing the correlation between input frames and information in the memory bank.

[0048] For example, see Figure 3 , Figure 3 This is a schematic diagram of a scene for image segmentation using the SAM2 model provided in an embodiment of the present application. Figure 3 As shown in the figure, the target location is inside the elevator. After obtaining the panoramic image of the elevator, the panoramic image of the elevator is segmented based on the SAM2 model. The SAM2 model can identify all object features in the panoramic image and circle them in the form of boxes. The feature objects identified in the boxes from left to right in the figure are person object, object object, person object, and person object respectively. The feature objects in the boxes can be further analyzed later.

[0049] It can be seen that in this example, the panoramic image information collected by the perception module is processed through preprocessing operations and image segmentation algorithms to achieve the acquisition, optimization and segmentation of panoramic image information, and then obtain multiple feature objects, thereby improving data quality and providing a rich and accurate input basis for subsequent tasks.

[0050] S202 : Determine whether there is a target free position in the target location according to the spatial features of the multiple feature objects, the spatial features of the biped robot, and the spatial features of the target location.

[0051] Among them, spatial features include occupied space data. Specifically, occupied space data can be obtained through occupancy matrix, point cloud data and bounding box information. The occupancy matrix divides the interior of the elevator into a two-dimensional matrix and records whether each position is occupied; point cloud data uses depth cameras and other devices to collect point cloud data inside the elevator. Areas without people will show fewer dense point groups, while areas with people will show more dense point groups; if there is a camera monitoring the interior of the elevator, the target detection algorithm can be used to identify any visible objects and obtain their bounding box information.

[0052] In one possible implementation, determining whether there is a target free position within the target location based on the spatial features of the multiple feature objects, the spatial features of the biped robot, and the spatial features of the target location includes:

[0053] Obtain first occupied space data of the multiple feature objects within the target location; obtain second occupied space data of the bipedal robot itself and the carried items, the second occupied space data being the data with the minimum occupied space obtained by changing the relative position of the bipedal robot itself and the carried items; obtain third occupied space data of the target location; obtain the difference between the third occupied space data and the first occupied space data, the difference being the free space data; when it is detected that the free space data is not less than the second occupied space data and that a target free position exists in the free space data, it is determined that a target free position exists in the target location; when it is detected that the free space data is less than the second occupied space data and / or that the target free position does not exist in the free space data, it is determined that no target free position exists in the target location.

[0054] Among them, the target idle position can be set by the user in the control module of the bipedal robot. The target idle position can also be any position in the target location that can accommodate the bipedal robot. For example, the target idle position is a position close to the elevator door and close to the elevator floor control panel.

[0055] Among them, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a scene before a bipedal robot enters an elevator provided in an embodiment of the present application. Figure 4 As shown, a target idle position A and an elevator floor control panel B are provided in the target location C. The bipedal robot 100 obtains the occupied space data in the target location C, and then determines whether the target idle position A can accommodate itself based on the occupied space data. Specifically, when the bipedal robot 100 determines whether there is a target idle position A in the target location C, it can first obtain the difference between the third occupied space data of the target location C and the first occupied space data of multiple other feature objects in the target location C. If the difference is greater than or equal to the minimum occupied space data obtained by changing the relative position of the bipedal robot 100 itself and the carried items, it is considered that the target idle position A exists in the target location C.

[0056] The user can set the default carrying method for the bipedal robot and its carried items, i.e., the default relative position between the bipedal robot and the carried items, in the bipedal robot's control module. When the bipedal robot reaches its destination, it can change its relative position to the carried items while ensuring the safety and integrity of the carried items, thereby reducing the space occupied by the bipedal robot. Specifically, when assigning the bipedal robot a delivery task via elevator, the user can pre-set the permitted carrying method for the required items, i.e., the permitted relative position between the bipedal robot and the carried items. For example, if the carried item is a cake, the user can set the bipedal robot to only be able to move the cake horizontally or lift it vertically, without allowing rotation, to ensure the cake remains intact.

[0057] It can be seen that in this example, by comprehensively considering multiple feature objects, the bipedal robot itself, the occupied space data of the carried items and the target location to determine whether there is a target free position, the intelligent decision-making of the bipedal robot is achieved.

[0058] S203: Detecting that there is a target vacant position in the target location, and determining a target behavior strategy according to the person recognition results of the multiple feature objects.

[0059] The target behavior strategy is used to complete the elevator navigation task of the bipedal robot.

[0060] Among them, the person recognition result can be the result of identifying multiple feature objects in the target location and judging whether there is a person, or the person recognition result can be the result of identifying the person object among multiple feature objects in the target location and judging whether there is a target person. Specifically, when judging whether there is a target person among multiple feature objects, face recognition or feature extraction can be performed on all identified people, and then further judgment can be made.

[0061] In a possible implementation, the multiple feature objects include person objects and object objects, and determining the target behavior strategy based on the person recognition results of the multiple feature objects includes:

[0062] Perform person recognition on the multiple feature objects to obtain a person recognition result; detect that the person recognition result is that a person object exists among the multiple feature objects, determine that the target behavior strategy is to output a first speech voice, navigate to the target idle position, and determine a first behavior strategy based on the elevator floor control panel image acquired by the perception module; detect that the person recognition result is that no person object exists among the multiple feature objects, determine that the target behavior strategy is to navigate to the target idle position, and press the target floor button on the elevator floor control panel.

[0063] Among them, the first speech voice can be preset by the user to remind the people in the target location that the bipedal robot is about to enter the elevator. For example, when the target location is an elevator, the first speech voice can be "Please note, the robot is about to enter the elevator, be careful to avoid it."

[0064] Among them, when the biped robot navigates to the target idle position, the path planning algorithm of the biped robot can be an A-star algorithm, a Dijkstra algorithm, a rapidly exploring random tree (RRT), and a potential field algorithm (Potential Field).

[0065] It can be seen that in this example, the target behavior strategies in different situations are determined by the character recognition results of multiple feature objects, which is conducive to improving the intelligence of the bipedal robot in practical applications.

[0066] In a possible implementation, determining a first behavior strategy based on the image of the elevator floor control panel acquired by the perception module includes:

[0067] Acquire an image of the elevator floor control panel captured by the perception module, the image of the elevator floor control panel being used to indicate a floor button in a selected state in the elevator floor control panel; determine whether a target floor button in the elevator floor control panel is in a selected state based on the image of the elevator floor control panel; detect that the target floor button in the elevator floor control panel is not in a selected state, determine a first behavior strategy as determining a target floor button pressing strategy based on an obstacle detection result between itself and the elevator floor control panel, determine that after the target floor button is in a selected state, maintain its own posture until it reaches the target floor, and determine a movement mode based on a detection result of a moving object in front of itself; detect that the target floor button in the elevator floor control panel is in a selected state, determine a first behavior strategy as maintaining its own posture until it reaches the target floor, and determine a movement mode based on a detection result of a moving object in front of itself.

[0068] The steps for pressing the target floor button on the elevator floor control panel are as follows: Use a visual perception module or sensor, such as a camera, to detect the elevator floor control panel inside the elevator and locate the target floor button. Analyze and process the captured image using image processing and computer vision technology to identify the target floor button, using algorithms such as object detection and feature matching to achieve target recognition. Based on the robot's current position and the location of the target floor button, path planning is performed to determine how to move to the target location. A navigation algorithm can be used to plan the robot's path. Based on the path obtained through motion planning, the robot is controlled to perform corresponding actions to move to the target location. When the robot reaches the target location, it accurately and stably presses the target floor button using a robotic arm or an indicator device in a predetermined posture.

[0069] To detect whether there is a moving object in front of the robot, different sensors and technologies can be used to obtain detection results. The method is as follows:

[0070] Moving object detection based on visual sensors: Bipedal robots can be equipped with visual sensors such as cameras or depth cameras. By capturing images or point cloud data in the environment for analysis, they can identify and track moving objects using computer vision algorithms (such as target detection and motion tracking).

[0071] Moving object detection based on LiDAR: LiDAR can emit laser beams and receive their reflected signals to obtain the distance and position information of surrounding objects. By continuously scanning the front area and analyzing the returned data, moving objects can be detected.

[0072] Moving object detection based on infrared sensors: Infrared sensors can detect infrared radiation generated by objects. By placing infrared sensors at appropriate positions in front of the robot, it can be determined whether there are moving objects passing by.

[0073] Moving object detection based on ultrasonic sensors: Ultrasonic sensors can send ultrasonic waves and receive their echoes to measure the distance to obstacles. By placing an ultrasonic sensor array on the front of the robot, it can detect moving objects that may exist in the front area.

[0074] As can be seen in this example, by acquiring and analyzing images of the elevator floor control panel, it is possible to accurately determine whether the target floor button is selected. This prevents abnormal behavior or unnecessary movement due to misjudgment or incorrect operation. Determining movement patterns based on the detection of moving objects in front of the robot allows the robot to flexibly adjust its movement based on environmental changes, which helps improve its intelligence.

[0075] In one possible implementation, determining a target floor button pressing strategy based on an obstacle detection result between the elevator and the elevator floor control panel includes:

[0076] Obtain an obstacle detection result between itself and the elevator floor control panel; detect that the obstacle detection result is that there is an obstacle between itself and the elevator floor control panel, and determine that the target floor button pressing strategy is to output a second speech voice, and the second speech voice is used to ask others to press the target floor button; detect that the obstacle detection result is that there is no obstacle between itself and the elevator floor control panel, and determine that the target floor button pressing strategy is to determine the target adjustment angle of each of its own action joints according to the spatial relationship between itself and the target button, so as to press the target floor button.

[0077] Obstacle detection results can be obtained through the following methods: Using distance sensors such as ultrasonic sensors, infrared sensors, or lidar installed on the front of the robot, the robot can detect the distance between the robot and objects (including elevator floor control panels). When the detected distance is less than a set threshold, an obstacle is determined to be present. Alternatively, visual sensors such as cameras or depth cameras can be used to capture images of the elevator floor control panel and perform object detection and segmentation using computer vision technology. By analyzing the image for obstructing objects or edge information, the presence of obstacles can be inferred.

[0078] Among them, see Figure 5 , Figure 5 FIG. 1 is a schematic diagram of a scene after a bipedal robot enters an elevator provided in an embodiment of the present application. Figure 5 As shown, after the bipedal robot 100 enters the target idle position A of the target location C, it determines that there is an obstacle person D between itself and the elevator floor control panel B based on the obstacle detection result. At this time, the bipedal robot 100 outputs the second speech voice through the microphone array carried by itself. The second speech voice is: "Hello, please press the X floor for me, thank you."

[0079] As can be seen in this example, the target floor button pressing strategy is determined based on the obstacle detection result between the elevator and the elevator floor control panel. When an obstacle is detected, a second voice message is output to request help from others to press the target floor button. This can intelligently handle situations where direct operation is impossible due to obstacles.

[0080] In one possible implementation, determining a movement mode based on a detection result of a moving object in front of the user includes:

[0081] Obtain a moving object detection result in front of itself; detect that the moving object detection result is that there is a moving object in front of itself, and determine that the moving mode is a following mode; detect that the moving object detection result is that there is no moving object in front of itself, and determine that the moving mode is an autonomous mode.

[0082] Among them, obtaining the detection result of the moving object in front of itself can be achieved by using a visual sensor, a lidar or other moving object detection technologies.

[0083] When the robot detects a moving object in front of it, it can enter follow mode. In this mode, the robot adjusts to the object's position and trajectory, maintaining a certain distance and following its movement. This mode is suitable for scenarios that require close attention and tracking of a specific target (such as a person or object), such as when following a human user during navigation.

[0084] When the robot detects no moving objects in front of it, it can enter autonomous mode. In this mode, the robot relies on pre-set path planning algorithms or environmental perception capabilities to independently navigate and move without relying on external targets for guidance. This mode is suitable for situations where there is no specific goal or independent tasks need to be completed, such as patrolling and exploration.

[0085] It can be seen that in this example, by determining different movement modes based on the mobile object detection results and selecting appropriate behavior strategies in different situations, the robot can respond to various environments more intelligently and flexibly.

[0086] In a possible implementation, the method further includes:

[0087] It is detected that there is no target idle position in the target location, and it is determined whether there is a person moving in the target location; it is detected that there is a person moving in the target location, and a target behavior strategy is determined to output a third speech voice, and after detecting that there is a target idle position in the target location, it navigates to the target idle position, and determines a second behavior strategy based on the elevator floor control panel image captured by the perception module; it is detected that there is no person moving in the target location, and the target behavior strategy is determined to wait for the next elevator; and the bipedal robot is controlled to execute the target behavior strategy.

[0088] Among them, when the occupied space data or image segmentation results obtained by the perception module detect that there is no target idle position, it further determines whether there is a person moving, and then continues to judge after the person moves to determine whether there is a target idle position in the target location.

[0089] Among them, the third voice of speech is used to inform the person in the target location that he is moving to the target vacant position, or the third voice of speech can be used to request the person in the target location to continue moving to make a certain space for himself. For example, the third voice of speech can be "I am entering the elevator, please pay attention to avoid it, thank you."

[0090] It can be seen that in this example, by detecting whether there is a target free position in the target location and whether there is a person moving, the target behavior strategy is determined according to different situations, which helps to make the robot more flexible and intelligent in practical applications.

[0091] In a possible implementation, after detecting that no person moves in the target location, the method further includes:

[0092] The target behavior strategy is determined to output a fourth speech voice, and after detecting that there is a target idle position in the target location, the target behavior strategy is navigated to the target idle position, and the third behavior strategy is determined based on the elevator floor control panel image collected by the perception module.

[0093] The fourth voice is used to request the characters in the target location to move to leave the target free position. For example, the fourth voice may be "Please move to make some room for me, thank you."

[0094] Among them, if after the fourth speech voice is output, it is detected that no person moves in the target location, or if a person moves and there is no target free position in the target location after the person moves, the target behavior strategy is determined to be waiting for the next elevator.

[0095] It can be seen that in this example, after detecting that no people are moving in the target location, outputting the fourth speech voice can effectively inform the user of the current situation or the next plan. The instant voice feedback can enhance the interaction between people and robots and improve the user experience.

[0096] In a possible implementation, the method further includes: determining whether the walking path width of the target location exceeds a first width threshold based on image information collected by the perception module; upon detecting that the walking path width exceeds the first width threshold, determining whether there is a target idle position within the target location based on the spatial characteristics of the multiple feature objects, the spatial characteristics of the bipedal robot, and the spatial characteristics of the target location; and when detecting that the walking path width does not exceed the first width threshold, determining the target behavior strategy to wait for the next elevator.

[0097] Specifically, the bipedal robot is equipped with sensors or cameras to monitor the width of the target travel path in real time. The sensors can measure the distance between obstacles on both sides and compare it with the robot's own size.

[0098] Among them, the first width threshold can be set by the user in the control module of the bipedal robot. For example, the target location is inside the elevator, and the first width threshold is 20 cm. Before the bipedal robot is ready to enter the elevator, it first detects whether the current elevator door is closing and whether the current width left by the elevator door is greater than 20 cm. If it is greater than 20 cm, it continues to make subsequent judgments; if the current width left by the elevator door is less than 20 cm, it is considered that entering the elevator is relatively dangerous, and the target behavior strategy is determined to be waiting for the next elevator.

[0099] It can be seen that in this example, judging and detecting the width of the walking path based on video information and determining the target behavior strategy of the bipedal robot according to the relationship between the width and the first width threshold can help improve the intelligence of the bipedal robot.

[0100] In a possible implementation, the user can also remotely control the bipedal robot through the client of the remote control system. For details, please refer to Figure 6 , Figure 6 This is a schematic diagram of a remote operation system terminal interface of a bipedal robot provided in an embodiment of the present application. Figure 6 As shown, the remote operation system terminal interface is used to display the real-time video collected by the bipedal robot through the visual sensor. A first operation button E and a second operation button F are respectively provided in the lower left corner and the lower right corner of the operation interface. The first operation button E is used to control the bipedal robot to turn left and right, and the second operation button F is used to control the bipedal robot to move forward and backward. A status bar G is provided in the upper right corner of the operation interface. The status bar G is used to display information such as the current time, communication connection status, and power level of the remote operation system terminal. A function button H is provided in the status bar G. By clicking the function button H on the operation interface, the bipedal robot mode adjustment interface can be opened in the middle of the operation interface. The user can change and set the mode of the bipedal robot, the display method of the real-time video collected by the bipedal robot through the visual sensor, the function of each operation button, the button layout of the operation interface, etc. on the bipedal robot mode adjustment interface.

[0101] Among them, see Figure 7 , Figure 7 : is a schematic diagram of a mode adjustment interface of a biped robot provided in an embodiment of the present application, such as Figure 7As shown, the user can now obtain the real-time posture of the bipedal robot on the left side of the mode adjustment interface and rotate the real-time posture of the bipedal robot by adjusting the button K set below the area for displaying the real-time posture of the bipedal robot. The parameter box J on the right side of the display interface of the teleoperation system terminal displays multiple parameters of the current bipedal robot, including but not limited to the position of each joint point of the bipedal robot, joint angles, the orientation and rotation angle of the bipedal robot, and the head status information of the bipedal robot. At the same time, the parameter box J includes the current X frame speed of the bipedal robot (the speed of the bipedal robot moving in the front and back directions on the plane) and the Y frame speed of the bipedal robot (the speed of the bipedal robot moving in the left and right directions on the plane). The user can directly adjust the above multiple parameters in the parameter box J. The current three-dimensional spatial coordinate position of the bipedal robot is usually represented by (x, y, z), and the current orientation and rotation angle of the bipedal robot can be described using Euler angles, quaternions, or rotation matrices.

[0102] S204: Control the bipedal robot to execute the target behavior strategy.

[0103] It can be seen that in this example, by performing image segmentation on the collected panoramic image, multiple detections are performed on the identified feature objects, and finally the robot's target behavior strategy is determined based on the detection results. The robot completes the elevator delivery task based on the conditions and makes self-service decisions without the need for assistance from other elevator passengers. This is conducive to improving the intelligence of the robot in performing elevator delivery tasks.

[0104] See also Figure 8 , Figure 8 This is a structural diagram of a bipedal robot task navigation device based on image segmentation provided in an embodiment of the present application, such as Figure 8 As shown, the biped robot task navigation device 800 based on image segmentation includes:

[0105] An acquisition module 801 is configured to acquire a plurality of feature objects obtained by performing image segmentation based on the panoramic image information collected at the target location by the perception module;

[0106] Processing module 802 is used to determine whether there is a target idle position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot and the spatial features of the target location; and to detect the existence of the target idle position in the target location and determine a target behavior strategy based on the character recognition results of the multiple feature objects, the target behavior strategy is used to complete the elevator task navigation of the bipedal robot; and to control the bipedal robot to execute the target behavior strategy.

[0107] In one possible implementation, in terms of obtaining multiple feature objects obtained by image segmentation based on the panoramic image information collected by the perception module at the target location, the acquisition module 801 is specifically used to: obtain the first panoramic image information collected by the perception module at the target location, the panoramic image information including human elements, object elements and background elements; obtain the second panoramic image information after the perception module performs a preprocessing operation on the first panoramic image information, the preprocessing operation including one or more of denoising, translation and rotation, image size adjustment, and brightness and contrast adjustment; obtain the multiple feature objects obtained by image segmentation of the second panoramic image information by the perception module according to the everything segmentation model 2 in the machine learning model.

[0108] In one possible implementation, the spatial features include occupied space data. In determining whether there is a target free position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot, and the spatial features of the target location, the processing module 802 is specifically used to: obtain first occupied space data of the multiple feature objects in the target location; obtain second occupied space data of the bipedal robot itself and the carried items, the second occupied space data being the data with the minimum occupied space obtained by changing the relative position of the bipedal robot itself and the carried items; obtain third occupied space data of the target location; obtain the difference between the third occupied space data and the first occupied space data, the difference being the free space data; when it is detected that the free space data is not less than the second occupied space data and the target free position exists in the free space data, determine that the target free position exists in the target location; when it is detected that the free space data is less than the second occupied space data and / or the target free position does not exist in the free space data, determine that there is no target free position in the target location.

[0109] In one possible implementation, multiple feature objects include person objects and object objects. In terms of determining the target behavior strategy based on the person recognition results of the multiple feature objects, the processing module 802 is specifically used to: perform person recognition on the multiple feature objects to obtain a person recognition result; detect that the person recognition result is that there is a person object among the multiple feature objects, determine that the target behavior strategy is to output a first speech voice, navigate to the target idle position, and determine the first behavior strategy based on the elevator floor control panel image acquired by the perception module; detect that the person recognition result is that there is no person object among the multiple feature objects, determine that the target behavior strategy is to navigate to the target idle position, and press the target floor button of the elevator floor control panel.

[0110] In one possible implementation, in terms of determining the first behavior strategy based on the image of the elevator floor control board acquired by the perception module, the processing module 802 is specifically used to: acquire the image of the elevator floor control board acquired by the perception module, the image of the elevator floor control board being used to indicate the floor button in the elevator floor control board that is in the selected state; determine whether the target floor button in the elevator floor control board is in the selected state based on the image of the elevator floor control board; detect that the target floor button in the elevator floor control board is not in the selected state, determine that the first behavior strategy is to determine the target floor button pressing strategy based on the obstacle detection result between itself and the elevator floor control board, maintain its own posture until it reaches the target floor after determining that the target floor button is in the selected state, and determine the movement mode based on the detection result of the moving object in front of itself; detect that the target floor button in the elevator floor control board is in the selected state, determine that the first behavior strategy is to maintain its own posture until it reaches the target floor, and determine the movement mode based on the detection result of the moving object in front of itself.

[0111] In one possible implementation, in terms of determining the target floor button pressing strategy based on the obstacle detection result between itself and the elevator floor control panel, the processing module 802 is specifically used to: obtain the obstacle detection result between itself and the elevator floor control panel; detect that the obstacle detection result is that there is an obstacle between itself and the elevator floor control panel, and determine that the target floor button pressing strategy is to output a second speech voice, and the second speech voice is used to ask others to press the target floor button; detect that the obstacle detection result is that there is no obstacle between itself and the elevator floor control panel, and determine that the target floor button pressing strategy is to determine the target adjustment angle of each of its own action joints based on the spatial relationship between itself and the target button, so as to press the target floor button.

[0112] In one possible implementation, in terms of determining the movement mode based on the detection result of the moving object in front of itself, the processing module 802 is specifically used to: obtain the detection result of the moving object in front of itself; detect that the moving object detection result is that there is a moving object in front of itself, and determine that the movement mode is a following mode; detect that the moving object detection result is that there is no moving object in front of itself, and determine that the movement mode is an autonomous mode.

[0113] In one possible implementation, after determining whether there is a target idle position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot, and the spatial features of the target location, the processing module 802 is further used to: detect that there is no target idle position in the target location, and judge whether there is a person moving in the target location; detect that there is a person moving in the target location, determine that the target behavior strategy is to output a third speech voice, navigate to the target idle position after detecting that there is a target idle position in the target location, and determine the second behavior strategy based on the elevator floor control panel image captured by the perception module; detect that there is no person moving in the target location, and determine that the target behavior strategy is to wait for the next elevator, and the third speech voice is used to inform the person in the target location that he or she is moving to the target idle position; control the bipedal robot to execute the target behavior strategy.

[0114] In one possible implementation, after detecting that no person is moving in the target location, the processing module 802 is further used to: determine that the target behavior strategy is to output a fourth speech voice, navigate to the target idle position after detecting that there is a target idle position in the target location, and determine a third behavior strategy based on the elevator floor control board image captured by the perception module, and the fourth speech voice is used to request the person in the target location to move to leave the target idle position.

[0115] It is worth noting that the specific functional implementation of the biped robot task navigation device 800 based on image segmentation can be found in the above Figure 2 The description of the bipedal robot task navigation method based on image segmentation is shown, for example, where the acquisition module 801 is used to implement the relevant content of executing S201, and the processing module 802 is used to implement the relevant content of executing S202-S204. The various units or modules in the bipedal robot task navigation device based on image segmentation 800 can be individually or completely combined into one or more other units or modules to form a structure, or one or more of the units or modules can be further divided into multiple functionally smaller units or modules to form a structure, which can achieve the same operation without affecting the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), and the functions of multiple units (or modules) are implemented by one unit (or module).

[0116] Based on the description of the above method embodiment and related device embodiment, please refer to Figure 9 , Figure 9 This is a structural diagram of a computer provided in an embodiment of the present application. Figure 9The computer 900 shown includes a processor 901 , a memory 902 , a communication interface 903 , and a bus 904 . The processor 901 , the memory 902 , and the communication interface 903 are communicatively connected to each other via the bus 904 .

[0117] Optionally, the memory 902 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0118] The memory 902 can store programs. When the executable program code stored in the memory 902 is executed by the processor 901, the processor 901 and the communication interface 903 are used to execute Figure 2 The illustrated embodiment shows various steps of the biped robot task navigation method based on image segmentation.

[0119] The processor 901 adopts a general-purpose CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to perform the biped robot task navigation method based on image segmentation of the method embodiment of the present application.

[0120] Processor 901 can also be an integrated circuit chip with signal processing capabilities. During implementation, the various steps of the image segmentation-based bipedal robot task navigation method of the present application can be completed by hardware integrated logic circuits or software instructions in processor 901. Optionally, processor 901 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor is a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The optional software module is located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 902, and the processor 901 reads the information in the memory 902, and combines its hardware to complete the functions required to be performed by the acquisition module 801 and the processing module 802 included in a bipedal robot task navigation device 800 based on image segmentation in an embodiment of the present application, or executes the bipedal robot task navigation method based on image segmentation in an embodiment of the method of the present application.

[0121] The communication interface 903 uses, for example but not limited to, a transceiver and other transceiver-related devices.

[0122] The bus 904 may include a path for transmitting information between various components of the computer 900 (eg, the memory 902 , the processor 901 , and the communication interface 903 ).

[0123] It should be noted that although Figure 9 The computer 900 shown only shows a memory, a processor, and a communication interface. However, in the specific implementation process, those skilled in the art should understand that the computer 900 also includes other devices necessary for normal operation. At the same time, based on specific needs, those skilled in the art should understand that the computer 900 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the computer 900 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 9 All devices shown in .

[0124] An embodiment of the present application provides a computer-readable storage medium, in which a computer program for electronic data exchange is stored. The computer program includes execution instructions, and the execution instructions are used to execute part or all of the steps of any one of the bipedal robot task navigation methods based on image segmentation as described in the above-mentioned embodiment of the bipedal robot task navigation method based on image segmentation. The above-mentioned computer includes an electronic terminal device.

[0125] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to enable a computer to perform part or all of the steps of any one of the image segmentation-based bipedal robot task navigation methods recorded in the above method embodiments. The computer program product can be a software installation package.

[0126] It should be noted that for any of the aforementioned embodiments of the bipedal robot task navigation method based on image segmentation, for the sake of simplicity, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that this application is not limited to the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0127] The above is a detailed introduction to the embodiments of the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application's method for task navigation of a bipedal robot based on image segmentation and related devices. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the idea of ​​the present application's method for task navigation of a bipedal robot based on image segmentation and related devices, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The memory may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0130] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps. The fact that certain measures are recited in different dependent claims does not mean that these measures cannot be combined to produce good results.

[0131] A person skilled in the art will understand that all or part of the steps in the various methods of any of the above-mentioned methods of the biped robot task navigation method based on image segmentation can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0132] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in an embodiment of a biped robot task navigation method based on image segmentation in this application, such as the device and computer program product in the above flowchart, falls within the scope of the related products described in this application.

[0133] Obviously, those skilled in the art may make various modifications and variations to the image segmentation-based bipedal robot task navigation method and related apparatus provided in this application without departing from the spirit and scope of this application. Thus, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is intended to include such modifications and variations.

Claims

1. A bipedal robot task navigation method based on image segmentation, characterized in that: The method is applied to a control module of the bipedal robot, wherein the bipedal robot includes the control module and a perception module in communication with the control module. Acquire multiple feature objects obtained by performing image segmentation based on panoramic image information collected by the perception module at the target location; determining whether there is a target free position in the target location according to the spatial features of the multiple feature objects, the spatial features of the biped robot, and the spatial features of the target location; Performing person recognition on the multiple feature objects to obtain a person recognition result; detecting that the person recognition result indicates that a person object exists among the multiple feature objects, determining a target behavior strategy as outputting a first speech voice, navigating to the target idle location, and determining a first behavior strategy based on an image of an elevator floor control panel acquired by the perception module; Detecting that the person recognition result indicates that no person object exists in the plurality of feature objects, determining a target behavior strategy as navigating to the target vacant location and pressing a target floor button on the elevator floor control panel; The bipedal robot is controlled to execute the target behavior strategy.

2. The method according to claim 1, wherein The acquiring of the plurality of feature objects obtained by performing image segmentation based on the panoramic image information collected by the perception module at the target location includes: Acquire first panoramic image information collected by the perception module at a target location, the panoramic image information including human elements, object elements, and background elements; Acquire second panoramic image information after the perception module performs a preprocessing operation on the first panoramic image information, where the preprocessing operation includes one or more of denoising, translation and rotation, image resizing, and brightness and contrast adjustment; The perception module is obtained to perform image segmentation on the second panoramic image information according to the object segmentation model 2 in the machine learning model to obtain multiple feature objects.

3. The method according to claim 1, wherein The spatial features include occupied space data, and determining whether there is a target free position in the target location based on the spatial features of the multiple feature objects, the spatial features of the biped robot, and the spatial features of the target location includes: Acquire first occupied space data of the plurality of feature objects in the target location; Acquire second occupied space data of the bipedal robot itself and the carried item, where the second occupied space data is data of the minimum occupied space obtained by changing the relative positions of the bipedal robot itself and the carried item; Acquiring third occupied space data of the target location; Obtaining a difference between the third occupied space data and the first occupied space data, where the difference is free space data; When it is detected that the free space data is not less than the second occupied space data and the target free position exists in the free space data, it is determined that the target free position exists in the target location; when it is detected that the free space data is less than the second occupied space data, and / or the target free position does not exist in the free space data, it is determined that the target free position does not exist in the target location.

4. The method according to claim 1, wherein The determining of the first behavior strategy based on the elevator floor control panel image acquired by the perception module includes: Acquire an elevator floor control panel image captured by the perception module, wherein the elevator floor control panel image is used to indicate a floor button in a selected state on the elevator floor control panel; determining whether a target floor button in the elevator floor control panel is in a selected state according to the elevator floor control panel image; detecting that the target floor button on the elevator floor control panel is not in a selected state, determining a first behavior strategy as determining a target floor button pressing strategy based on an obstacle detection result between the vehicle and the elevator floor control panel, maintaining the vehicle's posture until reaching the target floor after determining that the target floor button is in a selected state, and determining a movement mode based on a moving object detection result in front of the vehicle; It is detected that the target floor button in the elevator floor control panel is in a selected state, and the first behavior strategy is determined to maintain its own posture until reaching the target floor, and the movement mode is determined according to the detection result of the moving object in front of itself.

5. The method according to claim 4, wherein The determining of the target floor button pressing strategy based on the obstacle detection result between the elevator and the elevator floor control panel includes: Obtaining obstacle detection results between itself and the elevator floor control panel; detecting that the obstacle detection result is an obstacle between the elevator and the elevator floor control panel, and determining that the target floor button pressing strategy is to output a second speech voice, where the second speech voice is used to ask for help from others to press the target floor button; The obstacle detection result is detected as no obstacle between itself and the elevator floor control panel, and the target floor button pressing strategy is determined as determining the target adjustment angle of each of its own action joints based on the spatial relationship between itself and the target floor button to press the target floor button.

6. The method according to claim 4, wherein The determining of the movement mode according to the detection result of the moving object in front of the vehicle includes: Get the detection results of the moving object in front of itself; Detecting that the moving object detection result is that there is a moving object in front of the vehicle, and determining that the moving mode is a following mode; It is detected that the moving object detection result is that there is no moving object in front of the vehicle, and the moving mode is determined to be the autonomous mode.

7. The method according to any one of claims 1 to 6, wherein: After determining whether there is a target free position in the target location based on the spatial features of the multiple feature objects, the spatial features of the biped robot, and the spatial features of the target location, the method further includes: detecting that there is no target free position in the target location, and determining whether there is a person moving at the target location; Detecting that a person is moving in the target location, determining that the target behavior strategy is to output a third speech voice, navigating to the target idle position after detecting that there is a target idle position in the target location, and determining a second behavior strategy based on the elevator floor control panel image captured by the perception module; detecting that no person is moving in the target location, determining that the target behavior strategy is to wait for the next elevator, and the third speech voice is used to inform the person in the target location that the person is moving to the target idle position; The bipedal robot is controlled to execute the target behavior strategy.

8. The method according to claim 7, wherein After detecting that no person moves in the target location, the method further includes: The target behavior strategy is determined to be outputting a fourth speech voice, navigating to the target idle position after detecting the existence of a target idle position in the target location, and determining a third behavior strategy based on the elevator floor control board image captured by the perception module, wherein the fourth speech voice is used to request the person in the target location to move to leave the target idle position.

9. A bipedal robot task navigation device based on image segmentation, characterized in that: Applied to a bipedal robot, the bipedal robot includes a perception module, and the device includes: An acquisition module, configured to acquire a plurality of feature objects obtained by performing image segmentation based on the panoramic image information collected by the perception module at the target location; A processing module is used to determine whether there is a target idle position in the target location based on the spatial features of the multiple feature objects, the spatial features of the bipedal robot and the spatial features of the target location; and is used to perform person recognition on the multiple feature objects to obtain a person recognition result; and is used to detect that the person recognition result is that there is a person object among the multiple feature objects, determine that the target behavior strategy is to output a first speech voice, navigate to the target idle position, and determine a first behavior strategy based on the elevator floor control panel image acquired by the perception module; and is used to detect that the person recognition result is that there is no person object among the multiple feature objects, determine that the target behavior strategy is to navigate to the target idle position and press the target floor button on the elevator floor control panel; and is used to control the bipedal robot to execute the target behavior strategy.

Images