Body function implementation method based on environment constraint information and related device

Abstract

The invention discloses a body function implementation method based on environment constraint information and a related device. The method comprises the following steps: displaying a function demonstration interface on a display screen of a user terminal; detecting a trigger operation on a target function demonstration control in the plurality of function demonstration controls; in response to the trigger operation, sending a scene detection request message to the robot with the body; and receiving a first scene detection response message from the robot with the body, the first scene detection response message comprising a first prompt message, and the first prompt message comprising a second prompt message. The first prompt information is used for indicating that environment constraint information of a target scene where the robot with the body is currently located can meet comprehensive condition requirements for demonstrating the target function; and sending a function demonstration instruction to the robot with the body according to the first prompt information. According to the method, the safety and intelligence of the body robot for executing the user instruction can be improved.

Description

Technical Field

[0001] This application relates to the field of robot control, and in particular, to a method for realizing an embodied function based on environmental constraint information and related devices. Background Art

[0002] Currently, embodied robots are widely used in fields such as industrial manufacturing, home service, laboratory applications, service industries, and medical assistance. However, the current decision-making ability of embodied robots is insufficient, which leads to the inability of embodied robots to accurately execute user instructions, may pose threats to the environment and personnel safety, and at the same time reduces the user experience. Summary of the Invention

[0003] Embodiments of this application provide a method for realizing an embodied function based on environmental constraint information and related devices to improve the safety and intelligence of embodied robots in executing user instructions.

[0004] In a first aspect, embodiments of this application provide a method for realizing an embodied function based on environmental constraint information, which is applied to a user terminal. The user terminal is communicatively connected to an embodied robot. The method includes: Display a function demonstration interface on the display screen of the user terminal. The function demonstration interface includes a plurality of function demonstration controls; Detect a trigger operation on a target function demonstration control among the plurality of function demonstration controls. The target function demonstration control is any one of the plurality of function demonstration controls determined by the user; In response to the trigger operation, send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot. The scene detection request message includes the target function corresponding to the target function demonstration control. The scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt information, which is used to indicate that the environmental constraint information of the current target scene of the embodied robot can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint condition requirements and enhanced safety constraint condition requirements. The basic safety constraint condition requirements refer to that the unobstructed activity space of the embodied robot in the environmental constraint information meets the reference activity space required by the target function. The enhanced safety constraint condition requirements refer to that the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold. The sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot executing the target function; Send a function demonstration instruction to the embodied robot according to the first prompt information, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0005] Among them, the determination process of whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function includes the following steps: Obtain the environmental constraint information of the target scenario where the embodied robot is currently located; Construct a virtual scenario corresponding to the target scenario where the embodied robot is currently located according to the environmental constraint information; Determine the target action corresponding to the target function, where the target action is used to indicate the technical actions that the embodied robot needs to perform to complete the target function; Determine whether there is an interaction action in the target action, where the interaction action is an action for the embodied robot to interact with an individual; If there is no interaction action in the target action, simulate the implementation process of the target action in the virtual scenario to obtain the image data when the embodied robot performs the target action in the virtual scenario; Determine whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function according to the image data.

[0006] Among them, the determination process of whether the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function includes the following steps: Obtain the environmental constraint information of the target scenario where the embodied robot is currently located; Determine the image information of each movable object in the target scenario according to the environmental constraint information; Determine whether there is a space influence object according to the image information, where the space influence object is used to indicate an object with uncertain behavior activities; If there is no such space influence object, determine that the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function; If there is a space influence object, analyze the constrained state of each space influence object to obtain the constrained situation of each space influence object; Determine the probability of sudden occupation of the task space of each space influence object according to the constrained situation of each space influence object; Determine whether the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function according to the probability of sudden occupation of the task space of each space influence object and the preset probability threshold.

[0007] Among them, the determination process of whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function includes the following steps: Obtain the environmental constraint information of the target scenario where the embodied robot is currently located; Determine whether there are target hazards according to the environmental constraint information, where the target hazards are used to indicate the conditions, objects or events that may potentially damage the embodied robot when demonstrating the target function; If there are the target hazards, determine the association degree between each target hazard and the target action, where the target action is used to indicate the technical actions that the embodied robot needs to execute to complete the target function; If the association degree between the target hazard and the target action is greater than the preset association degree, determine that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function; If there is no association degree between the target hazard and the target action that is greater than the preset association degree, determine that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0008] Among them, the determination process of whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function further includes the following steps: If there are interactive actions in the target action, determine the matching degree between each movable object in the environmental constraint information and the interactive action, and obtain M matching degrees; Obtain the matching degrees that are greater than or equal to the preset matching degree among the M matching degrees, and obtain N target matching degrees, where N is less than or equal to M; If N is zero, determine that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function; If N is not zero, determine the movable object corresponding to each target matching degree among the N target matching degrees as a candidate interactive object, and obtain N candidate interactive objects; Determine the action completion degree between each candidate interactive object and the embodied robot in the virtual scenario, and obtain N action completion degrees; In the case where the highest action completion degree among the N action completion degrees is greater than or equal to the preset completion degree, determine that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0009] Among them, after sending the scene detection request message to the embodied robot, the method further includes: Receive a second scene detection response message from the embodied robot, where the second scene detection response message includes second prompt information for indicating that the environmental constraint information of the target scene where the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function; Send an environmental adjustment request message to the embodied robot according to the second prompt information; and receive an environmental adjustment response message from the embodied robot, where the adjustment response message includes third prompt information and a target environmental adjustment plan, and the third prompt information is used to indicate that a target result can be obtained by adjusting the environmental constraint information through the target environmental adjustment plan, and the target result is used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function; Send an environmental adjustment instruction to the embodied robot according to the third prompt information and the target environmental adjustment plan; and receive an environmental adjustment feedback message from the embodied robot; where the environmental adjustment feedback message includes fourth prompt information for indicating that the embodied robot has completed the adjustment of the environmental constraint information according to the target environmental adjustment plan; Send the function demonstration instruction to the embodied robot according to the fourth prompt information.

[0010] Wherein, the determination process of the third prompt information and the target environmental adjustment plan includes the following steps: Determine the reason why the environmental constraint information of the target scene where the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function according to the image data and / or the constrained situation of each spatial influence object; Determine at least one reference environmental adjustment plan according to the reason; Determine the feasibility of each reference environmental adjustment plan in the at least one reference environmental adjustment plan; If there is a reference environmental adjustment plan with a feasibility greater than the preset feasibility, determine the reference environmental adjustment plan with the highest feasibility as the target environmental adjustment plan; and generate the third prompt information according to the target environmental adjustment plan.

[0011] In a second aspect, an embodiment of the present application provides an embodied function implementation device based on environmental constraint information, which is applied to a user terminal, and the user terminal is communicatively connected to an embodied robot. The device includes: A display unit for displaying a function demonstration interface on the display screen of the user terminal, and the function demonstration interface includes a plurality of function demonstration controls; A detection unit, configured to detect a triggering operation on a target function demonstration control among the multiple function demonstration controls, where the target function demonstration control is any one of the multiple function demonstration controls determined by the user; A processing unit, configured to respond to the triggering operation, send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot, where the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt information, and the first prompt information is used to indicate that the environmental constraint information of the current target scene where the embodied robot is located can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint condition requirements and enhanced safety constraint condition requirements. The basic safety constraint condition requirements refer to that the unobstructed activity space of the embodied robot in the environmental constraint information meets the reference activity space required by the target function. The enhanced safety constraint condition requirements refer to that the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold. The sudden occupation of the task space means that the movable object occupies the reference activity space during the process of the embodied robot executing the target function; A sending unit, configured to send a function demonstration instruction to the embodied robot according to the first prompt information, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory storing action execution instructions, where the memory stores one or more programs; when the processor executes the action execution instructions stored in the memory, the processor executes the method described in the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, storing an energy data management program, including action execution instructions. When a processor of an electronic device executes the action execution instructions, the processor executes the method described in the first aspect.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, where the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0015] As can be seen, in the embodiment of the present application, first, a function demonstration interface is displayed on the display screen of the user terminal, and the function demonstration interface includes a plurality of function demonstration controls; then, a trigger operation on a target function demonstration control among the plurality of function demonstration controls is detected, and the target function demonstration control is any one of the plurality of function demonstration controls determined by the user; after that, in response to the trigger operation, a scene detection request message is sent to the embodied robot; and, a first scene detection response message from the embodied robot is received, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt message, and the first prompt message is used to indicate that the environmental constraint information of the current target scene where the embodied robot is located can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint condition requirements and enhanced safety constraint condition requirements. The basic safety constraint condition requirements refer to that the unobstructed activity space of the embodied robot in the environmental constraint information meets the reference activity space required by the target function. The enhanced safety constraint condition requirements refer to that the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold. The sudden occupation of the task space means that the movable object occupies the reference activity space during the process of the embodied robot executing the target function; finally, a function demonstration instruction is sent to the embodied robot according to the first prompt message, and the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0016] After the user terminal of the present application receives a user instruction, compared with directly controlling the robot to execute the user instruction, this solution takes into account the influence of the current environmental factors, sends a scene detection request to the robot, and analyzes through the robot whether there is an activity space corresponding to the target function and the probability that the activity space may be occupied in the current scene, so as to determine whether the target function can be demonstrated in the current environment. When the current environment can meet the requirements for demonstrating the target function, a function demonstration instruction is sent to the robot, enabling the robot to demonstrate the target function, thereby improving the safety and intelligence of the robot in executing the user instruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 is the system architecture diagram of a control system provided by an embodiment of the present application; Figure 2 is the schematic flowchart of an embodiment of the present application for implementing an embodied function based on environmental constraint information; Figure 3 is the schematic diagram of an embodiment of the present application for an embodied robot selection interface; Figure 4 is the schematic diagram of an embodiment of the present application for a function demonstration interface; Figure 5 is the schematic diagram of an embodiment of the present application for an outdoor scene; Figure 6 is the functional unit composition block diagram of an embodiment of the present application for an embodied function implementation device based on environmental constraint information; Figure 7 is the functional unit composition block diagram of another embodiment of the present application for an embodied function implementation device based on environmental constraint information; Figure 8 is the schematic diagram of the structure of an electronic device proposed by an embodiment of the present application. Detailed implementation manners

[0019] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0020] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0021] References to "embodiments" in this specification mean that the particular features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0022] Current embodied robots lack sufficient decision-making capabilities, which results in their inability to accurately execute user instructions, potentially posing threats to the environment and personnel safety, and at the same time degrading the user experience.

[0023] In view of the above problems, the embodiments of the present application provide a method and related device for realizing embodied functions based on environmental constraint information. The embodiments of the present application will be introduced in detail below with reference to the accompanying drawings.

[0024] Please refer to Figure 1 , Figure 1 which is a system architecture diagram of a control system provided by an embodiment of the present application. As Figure 1 shown, the control system 100 includes a user terminal 101 and a first embodied robot 102. The user terminal 101 can communicate with the first embodied robot 102 through a network or other connection means. The user terminal 101 refers to the device used by the user. The user can remotely control the embodied robot through the application program interface installed on the user terminal 101 to conduct function demonstrations, thereby generating function demonstration instructions. The generated function demonstration instructions will be sent from the user terminal 101 to the first embodied robot 102. The user terminal may include a tablet computer, a handheld computer, an in-vehicle electronic device, a server, a laptop computer, a mobile Internet electronic device (MID, Mobile Internet Devices), or a wearable electronic device (such as a smart watch, a Bluetooth headset), a smart phone (such as an Android phone, an iOS phone, a Windows Phone, etc.). The above are only examples, not an exhaustive list, and include but are not limited to the above electronic devices. The first embodied robot 102 analyzes the current scene of the embodied robot based on the received function demonstration instructions to determine whether the current scene is suitable for conducting a function demonstration, and sends the result to the user terminal 101. Among them, the first embodied robot 102 embeds artificial intelligence into a tangible entity such as a robot, enabling it to have the ability to perceive, learn, and dynamically participate in the surrounding environment. It can have a human-like appearance, including two legs, two arms, and a head. It is a robot that mimics human functions and intelligence and can perform various tasks in the working and living environments of humans.

[0025] Please refer to Figure 2 , Figure 2It is a schematic flowchart of an embodiment of the present application for implementing an embodied function based on environmental constraint information. This control method is applied to a user terminal, which is communicatively connected to an embodied robot. The method includes the following steps.

[0026] S210, display a function demonstration interface on the display screen of the user terminal.

[0027] Among them, when a function demonstration request for the embodied robot is detected, a function demonstration interface is displayed on the display screen. The function demonstration interface includes multiple function demonstration controls.

[0028] Among them, the user terminal can be bound to multiple embodied robots. Please refer to Figure 3 , Figure 3 It is a schematic diagram of an embodied robot selection interface provided by an embodiment of the present application. As shown in Figure 3 , before the user sends a function demonstration request, an embodied robot selection interface will be displayed on the display screen of the user terminal. The embodied robot selection interface includes multiple embodied robots bound to the user terminal, including the second embodied robot 1, the third embodied robot 2, the fourth embodied robot 3, etc. The second embodied robot 1, the third embodied robot 2, and the fourth embodied robot 3 refer to the names of these embodied robots. There is configuration prompt information in the upper left corner of the embodied robot selection interface to prompt the user to select the embodied robot to be demonstrated. There is image data of the configured embodied robot next to the name of the embodied robot for the user's reference. If the user clicks on the control of the second embodied robot 1, an enlarged version of the image information of this embodied robot will be displayed on the right side of the interface. When the user clicks the OK control, the user terminal enters the function demonstration interface. Please refer to Figure 4 , Figure 4 It is a schematic diagram of a function demonstration interface provided by an embodiment of the present application. As shown in Figure 4 , the function demonstration interface includes multiple function demonstration controls. The function demonstration controls can include handshake, squat, bend, somersault, pour water, take out the trash, etc. The user can control the movement of the embodied robot through the direction control panel 401 and the speed control panel 402, and control the actions of the embodied robot by clicking on the skill demonstration controls. The virtual space 403 is used to display the simulation result of the fifth embodied robot 404 according to the demonstration function selected by the user. This simulation result is sent by the fifth embodied robot 404.

[0029] S220, detect a trigger operation on the target function demonstration control among the multiple function demonstration controls.

[0030] Among them, the target function demonstration control is any one of the multiple function demonstration controls determined by the user.

[0031] Among them, when the user clicks on the demonstration function control, it corresponds to the triggering operation of the corresponding function space. Exemplarily, if the user clicks on the control for squatting, the corresponding target function control is the squatting control, indicating that the user needs the embodied robot to execute the instruction of squatting.

[0032] S230, in response to the triggering operation, send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot.

[0033] Among them, the scene detection request message includes the target function corresponding to the target function demonstration control. The scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt information, which is used to indicate that the environmental constraint information of the current target scene where the embodied robot is located can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint condition requirements and enhanced safety constraint condition requirements. The basic safety constraint condition requirements refer to that the unobstructed activity space of the embodied robot in the environmental constraint information meets the reference activity space required by the target function. The enhanced safety constraint condition requirements refer to that the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold. The sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot executing the target function.

[0034] Among them, after the user clicks on the demonstration function control, the user terminal transmits the function corresponding to the control clicked by the user to the embodied robot. After receiving the scene detection request, the embodied robot obtains scene pictures at various angles through the camera, and then analyzes the feasibility of function demonstration based on the scene pictures. Exemplarily, if the user clicks on the somersault control, the embodied robot determines whether the current environment is suitable for doing a somersault to determine whether the embodied robot needs to execute the instruction of doing a somersault.

[0035] Specifically, the process of determining the first prompt information includes the following steps: The process of determining whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function includes the following steps: Obtain the environmental constraint information of the target scenario where the embodied robot is currently located; construct a virtual scenario corresponding to the target scenario where the embodied robot is currently located according to the environmental constraint information; determine the target action corresponding to the target function, where the target action is used to indicate the technical actions that the embodied robot needs to perform to complete the target function; determine whether there is an interaction action in the target action, where the interaction action is an action for the embodied robot to interact with an individual; if there is no interaction action in the target action, then simulate the implementation process of the target action in the virtual scenario to obtain the image data when the embodied robot executes the target action in the virtual scenario; determine whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function according to the image data.

[0036] Among them, the embodied robot obtains the environmental information of the current location through a camera configured on the embodied robot. Further, it can be video data from multiple angles. Exemplarily, when the embodied robot receives a scene detection request message, it can rotate omnidirectionally to collect environmental information from various angles through the camera, and thus construct a virtual scenario. Among them, the virtual scenario is a three-dimensional graphical scenario constructed according to the three-dimensional spatial information of the real-time collected scene.

[0037] Among them, the target function refers to the specific effect to be achieved, and the target action is the specific ability required to achieve the effect. Exemplarily, if the target function is to pour water, the corresponding target actions include: finding a suitable container, placing the container in a stable position or holding it in the hand so that it will not tip over or move during the pouring process. Then, pick up the container filled with water, ensure a firm grip on the handle or neck of the container to maintain stability and control. Next, slowly tilt the container so that the water starts to flow out. Adjust the tilt angle of the container according to the amount of water to be poured out, and determine the water flow rate according to the capacity. And always observe the water level in the receiving container. When the water level reaches the required amount, stop pouring water in time. After pouring water, put the empty container back to its original position, ensure that the container is placed firmly to avoid rolling or collision. If there is water splashing or overflowing during the pouring process, it is necessary to wipe it clean with a tissue or a cloth in time to keep the tabletop or the ground clean.

[0038] After determining the target action, the embodied robot can also determine the category of the target action, that is, determine whether the target action is an interactive action or a solo demonstration action, and determine whether interaction with other individuals is required. If no interaction is required, it is determined that the target action is a solo demonstration action. Then, in the created virtual scene, the functions are simulated according to the position of the embodied robot to obtain video data of the embodied robot completing the target function in the scene. According to this video data, it is determined whether the current environment meets the requirements of the basic safety constraint conditions, that is, whether the obstacle-free activity space of the embodied robot meets the reference activity space required for the target function. Among them, this video data may show that the embodied robot can accurately execute the expected function, that is, the current environment meets the requirements of the basic safety constraint conditions; it may also show that the embodied robot can execute part of the expected function, but there are deviations or errors in some cases, and unexpected situations occur, that is, the current environment meets the requirements of the basic safety constraint conditions. For example, there are obstacles in the reference activity space, collisions occur with the obstacles, or the embodied robot performs a somersault action, but there is a puddle ahead, causing the embodied robot to fall.

[0039] It can be seen that in this embodiment, demonstrating the functions of the embodied robot in a virtual environment can avoid collisions, damages, or adverse effects on the environment that may occur during the actual execution process, and can predict and identify potential errors and problems, which helps to ensure the safety and intelligence of the embodied robot when operating in a real environment.

[0040] Specifically, the process of determining whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function further includes the following steps: If there are interactive actions in the target action, determine the matching degree between each movable object in the environmental constraint information and the interactive action to obtain M matching degrees; obtain the matching degrees greater than or equal to the preset matching degree among the M matching degrees to obtain N target matching degrees, where N is less than or equal to M; if N is zero, determine that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function; if N is not zero, determine the movable object corresponding to each target matching degree among the N target matching degrees as a candidate interactive object to obtain N candidate interactive objects; determine the action completion degree between each candidate interactive object and the embodied robot in the virtual scene to obtain N action completion degrees; in the case where the highest action completion degree among the N action completion degrees is greater than or equal to the preset completion degree, determine that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0041] Among them, if it is determined that the category of the target action is an interactive action, it is possible to determine whether there are movable objects in the current scene based on the scene picture. If there are no movable objects, a prompt message is sent to the user terminal to prompt the user that the interactive object does not exist and the target function cannot be demonstrated in the current scene.

[0042] Among them, if there are movable objects, obtain multiple movable objects existing in the current environmental constraint information, including children, young people, the elderly, pets, other robots, etc. Movable objects are used to indicate individuals that can move autonomously. Further, the category of the interactive object can be determined according to the interactive action, such as human, animal or other robots. Screen the multiple movable objects according to the determined category and interactive action. Exemplarily, if the interactive action is a handshake, animals and robots without arms can be screened out to obtain multiple reference interactive objects.

[0043] Among them, analyze and identify through the images of each reference interactive object to determine the matching degree of each reference interactive object with the interaction action. Specifically, it can be to identify the user's age, physical characteristics, characteristics related to the interaction action, etc. Physical characteristics refer to extracting key point data of the human body from the image. These key points usually include positions such as joints, head, and hands. Characteristics related to the interaction action include the posture of the human body, objects interacting with people, etc. Exemplarily, if the user instruction is a handshake, analyze the images of each interactive object, focusing on the hand, to identify whether it is convenient to perform a handshake operation. If the first interactive object is holding a child, the matching degree is low, then it is determined that the handshake operation cannot be performed. If the second interactive object is holding a drink in one hand and a shopping bag in the other hand, the matching degree is low, then it is determined that the handshake operation cannot be performed. If the third interactive object has both hands free, the matching degree is high, then it is determined as a candidate interactive object. If the fourth interactive object is holding a mobile phone in one hand and the other hand is free, the matching degree is relatively high, then it is determined as a candidate interactive object.

[0044] Among them, after obtaining the candidate interaction objects, the scores of each candidate interaction object can also be determined through indicators, such as emotion recognition, body feature analysis, position distance, etc., and the weighted sum of each indicator is calculated to obtain the score of each candidate interaction object. Then, according to the positions of the candidate interaction objects and the target actions, action simulation is performed in the virtual scene to predict the completion degree of the target actions. Based on the completion degree and the scores of the candidate interaction objects, the target interaction object is determined, and whether the target function can be achieved in the current scene is determined according to the completion degree. Exemplarily, the score of the first candidate interaction object is 89 points, the predicted completion degree is 90, the score of the second candidate interaction object is 93 points, the predicted completion degree is 87, the score of the third candidate interaction object is 91, and the predicted completion degree is 94. Assuming that the weight of the score is 0.3 and the weight of the predicted completion degree is 0.7, the total score of the first candidate interaction object is 89.7, the total score of the second candidate interaction object is 88.8, and the total score of the third candidate interaction object is 93.1. Then, the third candidate interaction object is determined as the target interaction object. And when the completion degree corresponding to the target interaction object is greater than or equal to the preset completion degree, it is determined that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function. When the completion degree corresponding to the target interaction object is less than the preset completion degree, it is determined that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0045] It can be seen that in the embodiments of the present application, by determining the interaction actions, interaction objects, and predicting the action completion degree, the target can be more accurately located, thereby more efficiently completing the task.

[0046] Specifically, the determination process of whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene where the embodied robot is currently located; determining whether there are target hazards according to the environmental constraint information, where the target hazards are used to indicate the conditions, objects, or events that may potentially cause damage to the embodied robot when demonstrating the target function; if there are the target hazards, determining the correlation degree between each target hazard and the target action, where the target action is used to indicate the technical action that the embodied robot needs to perform to complete the target function; if the correlation degree between the target hazard and the target action is greater than the preset correlation degree, determining that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function; if the correlation degree between the target hazard and the target action is not greater than the preset correlation degree, determining that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0047] Among them, the motion range of the embodied robot can be determined according to the target action, and it can be determined whether the current environment can satisfy the free movement of the embodied robot within this motion range. If it is satisfied, then it is further determined whether there are potential hazards in the current environment. The potential hazards can be obstacles in the scene, such as an unstable hot water kettle, plants, charging cables, etc., or potential hazards in the path, such as the flatness and friction of the ground. Then, the relevance between multiple potential hazards and the current target skill is determined. If the relevance is greater than the preset relevance, it is determined that the current environment cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function. For example, the potential hazards in the current environment include an obstacle green plant on the right and a puddle in the front. At this time, the "somersault" task is being executed. Since the target action of the "somersault" task is forward, the relevance with the puddle is high, while the relevance of the obstacle green plant is low. Since there is a target potential hazard puddle with a high relevance in the current environment, it is determined that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function, and this result is sent to the device terminal.

[0048] It can be seen that in the embodiments of the present application, by determining whether there are risk factors in the current environment, the safety of the embodied robot can be improved, the task execution can be optimized, and the intelligence of the embodied robot for executing user instructions can be enhanced.

[0049] Specifically, the determination process of whether the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene where the embodied robot is currently located; determining the image information of each movable object in the target scene according to the environmental constraint information; determining whether there is a space-influencing object according to the image information, and the space-influencing object is used to indicate an object with uncertainty in behavior activities; if there is no such space-influencing object, it is determined that the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function; if there is a space-influencing object, analyze the constrained state of each space-influencing object to obtain the constrained situation of each space-influencing object; determine the probability of sudden occupation of the task space of each space-influencing object according to the constrained situation of each space-influencing object; determine whether the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function according to the probability of sudden occupation of the task space of each space-influencing object and the preset probability threshold.

[0050] Among them, the age range, the category to which each movable object belongs, and the distance between each movable object and the reference activity space are determined according to the image information of each movable object. Multiple movable objects are screened according to the age range, the category to which they belong, and the distance to obtain the space-influencing objects. Specifically, they can be children under 12 years old, all pets, and humans whose distance from the reference activity space is less than a preset distance. The specific screening conditions are not limited herein. After that, the constrained situation of each space-influencing object is obtained. Exemplarily, it can be whether a child is constrained, that is, whether being led by a parent, or sitting in a vehicle that the child can control the movement, or sitting in a vehicle that the child cannot control the movement. It can also be whether a pet is constrained, that is, whether being on a leash, and whether the owner of the pet can hold back the pet. Specifically, the breed of the pet can be identified, the personality of the breed can be obtained, and whether the owner can hold back the pet can be determined according to the personality and the size of the owner. The probability of sudden occupation of the task space by each space-influencing object is determined according to the constrained situation and the behavior state of each space-influencing object. For example, it can be a 10-year-old child, not being led by a parent, with a calm mood, and being far from the reference activity space, then the probability of sudden occupation of the task space by this space-influencing object is relatively low.

[0051] Specifically, the image data of the movable object can also be input into the interference probability model to output the spatial influence object and the probability of the sudden occupation of the task space by the spatial influence object. The training process of the interference probability model includes: First, collect the image data of each of the multiple movable objects, which can be achieved through sensors, cameras, or other monitoring devices. For example, cameras can be installed to monitor the position, behavior, and emotional state of each movable object. At the same time, sensors can also be used to detect other potential interference sources in the environment, such as sound, light, etc. Then, determine the basic information of each movable object from the collected images, such as the age, gender, height, weight, etc. of children; the species, body type, personality, etc. of pets; the age, gender, height, weight, etc. of pet owners; and their behavior patterns, activity ranges, interaction methods with other objects, etc. Then, determine the spatial influence object based on this basic information, and extract the features related to the constrained situation of the spatial influence object. For example, whether a child is being led by a parent, whether a pet is on a leash, etc. Use machine learning or deep learning algorithms to train the extracted features to establish an interference probability model. During the training process, historical data or simulated data can be used as input, and the prediction performance can be optimized by continuously adjusting the model parameters. At the same time, expert knowledge or domain knowledge can also be considered to guide the training process of the model. In practical applications, new data needs to be continuously collected and the model updated to adapt to environmental changes. At the same time, the model also needs to be adjusted and optimized according to the actual effects to improve the prediction accuracy. Through continuous iteration and improvement, the dynamic algorithm can more accurately predict the spatial influence object and output the probability of the sudden occupation of the task space by the spatial influence object.

[0052] Among them, if the probability of the sudden occupation of the task space by the spatial influence object is greater than or equal to the preset probability threshold, it is determined that the current environmental constraint information cannot meet the enhanced safety constraint condition requirements of the demonstration target function; in the case where the probability of the sudden occupation of the task space by the spatial influence object is less than the preset probability threshold, it is determined that the current environmental constraint information can meet the enhanced safety constraint condition requirements of the demonstration target function.

[0053] It can be seen that in this embodiment, by determining the probability of the sudden occupation of the task space based on the constrained situation of the spatial influence object and predicting what may happen, potential problems can be discovered in a timely manner to protect the safety of on-site personnel and robots.

[0054] Specifically, after sending the scenario detection request message to the embodied robot, the method further includes: receiving a second scenario detection response message from the embodied robot, where the second scenario detection response message includes second prompt information for indicating that the environmental constraint information of the target scenario where the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function; sending an environment adjustment request message to the embodied robot according to the second prompt information; and receiving an environment adjustment response message from the embodied robot, where the adjustment response message includes third prompt information and a target environment adjustment plan, and the third prompt information is used to indicate that the target result can be obtained by adjusting the environmental constraint information through the target environment adjustment plan, and the target result is used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function; sending an environment adjustment instruction to the embodied robot according to the third prompt information and the target environment adjustment plan; and receiving an environment adjustment feedback message from the embodied robot; where the environment adjustment feedback message includes fourth prompt information for indicating that the embodied robot has completed the adjustment of the environmental constraint information according to the target environment adjustment plan; sending the function demonstration instruction to the embodied robot according to the fourth prompt information.

[0055] Among them, if the embodied robot determines that the environmental constraint information of the target scenario where it is currently located cannot meet the comprehensive condition requirements for demonstrating the target function and feeds it back to the user terminal, the user terminal sends an adjustment request to the embodied robot, and the embodied robot determines whether the current environment can be adjusted to achieve the target function. Further, it can be to adjust the position or direction of the embodied robot, or the position of the obstacle, etc. According to the initial position and initial direction of the embodied robot, as well as the simulated image data and the determined target hidden danger, it is determined whether there is an adjustment plan. If an adjustment plan is obtained, the feasibility of each adjustment plan is determined, and the adjustment plan with the highest feasibility is determined as the target adjustment plan, and the target adjustment plan is sent back to the user terminal, so that the user terminal issues an adjustment instruction to the embodied robot according to the target adjustment plan, so that the embodied robot makes adjustments according to it, and sends feedback information to the user terminal after the adjustment is completed. After receiving the feedback information, the user terminal can first determine whether the adjustment of the embodied robot is accurate. On the premise of accurate adjustment, a function demonstration instruction is issued, so that the embodied robot demonstrates the target function in the real scenario; if no adjustment plan is obtained, information indicating that the target function cannot be realized in the current scenario is prompted on the function demonstration interface, along with the reason.

[0056] Specifically, the determination process of the third prompt information and the target environment adjustment plan includes the following steps: determining the reason why the environmental constraint information of the target scenario where the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function according to the image data and / or the constrained situation of each spatial influence object; determining at least one reference environment adjustment plan according to the reason; determining the feasibility of each reference environment adjustment plan in the at least one reference environment adjustment plan; if there is a reference environment adjustment plan with a feasibility greater than the preset feasibility, determining the reference environment adjustment plan with the highest feasibility as the target environment adjustment plan; and generating the third prompt information according to the target environment adjustment plan.

[0057] Among them, the reason for non - satisfaction is determined according to the simulated video data and / or the constrained situation of each spatial influence object. Specifically, if the basic safety constraint condition requirements are not met, the reason for non - satisfaction is determined according to the simulated video data or the correlation degree of the target hidden danger; if the enhanced safety constraint condition requirements are not met, the reason for non - satisfaction is determined according to the constrained situation of each spatial influence object. For example, if the basic safety constraint condition requirements are not met, when performing a "somersault" task at the current position and current direction, the landing position is a puddle, which is likely to cause instability when standing, or when performing a somersault task at the current position and current direction, a table will be hit midway, causing the table to fall or damage the embodied robot. Then, it is determined whether the reason for non - satisfaction can be avoided. If it can be avoided, at least one adjustment plan is generated. For example, if there is a puddle in front of the embodied robot, it will fall into the puddle when performing the "somersault" skill. Then, based on the current environment, it is determined whether the robot can move forward, backward, rotate the direction, or move to other positions. If it can, based on the landing position of the robot's "somersault", the movement path is determined. After moving to the corresponding position, the "somersault" task is performed again. Exemplarily, the first adjustment plan can be to rotate 30 degrees clockwise to avoid the direction of the puddle; the second adjustment plan can be to move forward 2 meters to cross the puddle. Then, the feasibility of each adjustment plan is calculated. If there is no adjustment plan with a feasibility greater than the preset feasibility, it is determined that there is no adjustment plan. If there is an adjustment plan with a feasibility greater than the preset feasibility, the adjustment plan corresponding to the highest feasibility is determined as the target adjustment plan and sent to the user terminal. Exemplarily, for the first adjustment plan, after rotating 30 degrees clockwise and then performing the somersault task, there is a table obstacle, so the feasibility is relatively low. For the second adjustment plan, moving forward 2 meters to cross the puddle and then performing the somersault task can safely demonstrate the target action, that is, the ground is stable and there are no obstacles, so the feasibility is relatively high and it is determined as the target adjustment plan.

[0058] It can be seen that in the embodiment of the present application, in the case that the target function cannot be demonstrated in the current scenario, by generating an adjustment plan to demonstrate the target function, the intelligence of the embodied robot during action demonstration is increased.

[0059] S240. Send a function demonstration instruction to the embodied robot according to the first prompt information.

[0060] Wherein, the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0061] Specifically, after receiving the first prompt information sent by the embodied robot, the user terminal sends a function demonstration instruction to the embodied robot, enabling the embodied robot to implement the function in the real scenario. Exemplarily, the user clicks on the user terminal to have the embodied robot demonstrate the "somersault" skill. When it is determined that the embodied robot can implement the "somersault" skill in the current scenario, the embodied robot executes the "somersault" task in the real scenario.

[0062] It can be seen that in the embodiment of the present application, after receiving the user instruction, compared with directly controlling the robot to execute the user instruction, this solution takes into account the influence of the current environmental factors. The terminal sends a scene detection request to the robot, and the robot analyzes whether there is an activity space corresponding to the target function and the probability that the activity space may be occupied in the current scene it is in, so as to determine whether the target function can be demonstrated in the current environment. When the current environment can meet the requirement of demonstrating the target function, a function demonstration instruction is sent to the robot, enabling the robot to demonstrate the target function, thereby improving the safety and intelligence of the robot when executing the user instruction.

[0063] In a possible embodiment, please refer to Figure 5 , Figure 5 is a schematic diagram of an outdoor scene provided by the embodiment of the present application, as Figure 5As shown, the sixth embodied robot 506 is in an outdoor entertainment scenario. The user selects the function that the current robot is to demonstrate on the operation interface. After the user clicks the function control for playing table tennis, the function of playing table tennis is sent to the embodied robot, and the corresponding actions for playing table tennis are determined. It is analyzed that playing table tennis is an interactive action, and the first interaction object 507, the second interaction object 508, and the third interaction object 509 are determined. At the same time, the current scene image is obtained, and the target interaction object is determined among the first interaction object 507, the second interaction object 508, and the third interaction object based on the scene image. Exemplarily, it may be detected that the first interaction object 507 is young, the second interaction object 508 has difficulty moving, and the third interaction object 509 has the highest matching degree, and it is determined as the target interaction object. Further, the sixth embodied robot 506 is controlled to move to the position of the target interaction object for an interaction invitation. After the target interaction object agrees, the simulation of playing table tennis in the current environment is carried out. It is determined that the stool 501 will interfere with the execution of the action, and the first interaction object 507 is relatively close to the table tennis table 505 and is prone to danger. A second prompt message is sent to the user terminal. After receiving this information, the user terminal sends an environmental adjustment request message to the embodied robot. The embodied robot generates an adjustment plan based on the reasons why the function cannot be realized. The adjustment plan may include prompting the first interaction object 507 to move to a farther range and the path for moving the stool 501. For example, the path may be to move to the position of the stool 501, lift the stool 501, pass by the first interaction object 507, and then place the stool 501 around the first lounge chair 502, or the second lounge chair 503, or the third lounge chair 504. Among them, when passing by the first interaction object 507, a prompt message is generated for the first interaction object 507 to inform the playing range and prompt the first interaction object 507 to move to a safe position outside the playing range. The safe position may be at the position of the first lounge chair 502, the second lounge chair 503, or the third lounge chair 504. When it is determined that the adjustment plan is feasible, it is sent to the user terminal, so that the user terminal sends an adjustment instruction to the sixth embodied robot 506, enabling the embodied robot to adjust and thus demonstrate the function of playing table tennis with the third interaction object 509.

[0064] Consistent with the above embodiments, please refer to Figure 6 , Figure 6 is a block diagram of the functional units of an embodied function implementation device based on environmental constraint information provided by an embodiment of the present application, as Figure 6As shown in the figure, the embodied function implementation device 60 based on environmental constraint information includes: a display unit 61, configured to display a function demonstration interface on the display screen of the user terminal, where the function demonstration interface includes a plurality of function demonstration controls; a detection unit 62, configured to detect a trigger operation on a target function demonstration control among the plurality of function demonstration controls, where the target function demonstration control is any one of the plurality of function demonstration controls determined by the user; a processing unit 63, configured to respond to the trigger operation and send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot, where the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt message, and the first prompt message is used to indicate that the environmental constraint information of the current target scene of the embodied robot can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint condition requirements and enhanced safety constraint condition requirements. The basic safety constraint condition requirements refer to that the unobstructed activity space of the embodied robot in the environmental constraint information meets the reference activity space required for the target function. The enhanced safety constraint condition requirements refer to that the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold. The sudden occupation of the task space means that the movable object occupies the reference activity space during the process of the embodied robot executing the target function; a sending unit 64, configured to send a function demonstration instruction to the embodied robot according to the first prompt message, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0065] Among them, the determination process of whether the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the current target scene of the embodied robot; constructing a virtual scene corresponding to the current target scene of the embodied robot according to the environmental constraint information; determining the target action corresponding to the target function, where the target action is used to indicate the technical action that the embodied robot needs to execute to complete the target function; determining whether there is an interaction action in the target action, where the interaction action is an action for the embodied robot to interact with an individual; if there is no interaction action in the target action, then simulating the implementation process of the target action in the virtual scene to obtain image data when the embodied robot executes the target action in the virtual scene; and determining whether the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function according to the image data.

[0066] Among them, the determination process of whether the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene where the embodied robot is currently located; determining the image information of each movable object in the target scene according to the environmental constraint information; determining whether there is a space influence object according to the image information, where the space influence object is used to indicate an object with uncertainty in behavior activities; if there is no such space influence object, determining that the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function; if there is a space influence object, analyzing the constrained state of each space influence object to obtain the constrained situation of each space influence object; determining the probability of sudden occupation of the task space of each space influence object according to the constrained situation of each space influence object; and determining whether the environmental constraint information can meet the requirements of the enhanced safety constraint conditions for demonstrating the target function according to the probability of sudden occupation of the task space of each space influence object and the preset probability threshold.

[0067] Among them, the determination process of whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene where the embodied robot is currently located; determining whether there is a target hazard according to the environmental constraint information, where the target hazard is used to indicate a condition, object or event that may potentially cause damage to the embodied robot when demonstrating the target function; if there is a target hazard, determining the correlation degree between each target hazard and the target action, where the target action is used to indicate the technical action that the embodied robot needs to execute to complete the target function; if the correlation degree between the target hazard and the target action is greater than the preset correlation degree, determining that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function; if the correlation degree between the target hazard and the target action is not greater than the preset correlation degree, determining that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0068] Among them, the determination process of whether the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function further includes the following steps: If there is an interaction action in the target action, determine the matching degree between each movable object in the environmental constraint information and the interaction action, and obtain M matching degrees; Obtain the matching degrees that are greater than or equal to the preset matching degree among the M matching degrees, and obtain N target matching degrees, where N is less than or equal to M; If N is zero, determine that the environmental constraint information cannot meet the requirements of the basic safety constraint conditions for demonstrating the target function; If N is not zero, determine the movable object corresponding to each target matching degree among the N target matching degrees as a candidate interaction object, and obtain N candidate interaction objects; Determine the action completion degree between each candidate interaction object and the embodied robot in the virtual scene, and obtain N action completion degrees; In the case that the highest action completion degree among the N action completion degrees is greater than or equal to the preset completion degree, determine that the environmental constraint information can meet the requirements of the basic safety constraint conditions for demonstrating the target function.

[0069] In a possible embodiment, after sending the scene detection request message to the embodied robot, the embodied function implementation device 60 based on the environmental constraint information is further specifically configured to: Receive a second scene detection response message from the embodied robot, where the second scene detection response message includes second prompt information, and the second prompt information is used to indicate that the environmental constraint information of the target scene where the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function; Send an environment adjustment request message to the embodied robot according to the second prompt information; And receive an environment adjustment response message from the embodied robot, where the adjustment response message includes third prompt information and a target environment adjustment plan, and the third prompt information is used to indicate that the target result can be obtained by adjusting the environmental constraint information through the target environment adjustment plan, and the target result is used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function; Send an environment adjustment instruction to the embodied robot according to the third prompt information and the target environment adjustment plan; And receive an environment adjustment feedback message from the embodied robot; Among them, the environment adjustment feedback message includes fourth prompt information, and the fourth prompt information is used to indicate that the embodied robot has completed the adjustment of the environmental constraint information according to the target environment adjustment plan; Send the function demonstration instruction to the embodied robot according to the fourth prompt information.

[0070] Among them, the determination process of the third prompt information and the target environment adjustment plan includes the following steps: determining the reason why the environmental constraint information of the target scenario where the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function according to the image data and / or the constrained conditions of each spatial influence object; determining at least one reference environment adjustment plan according to the reason; determining the feasibility of each reference environment adjustment plan in the at least one reference environment adjustment plan; if there is a reference environment adjustment plan with a feasibility greater than the preset feasibility, determining the reference environment adjustment plan with the highest feasibility as the target environment adjustment plan; and generating the third prompt information according to the target environment adjustment plan.

[0071] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, therefore, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part, and will not be elaborated here.

[0072] In the case of adopting an integrated unit, please refer to Figure 7 , Figure 7 which is the functional unit composition block diagram of another embodied function implementation device based on environmental constraint information provided by the embodiment of the present application. As Figure 7 shown, the embodied function implementation device 60 based on environmental constraint information includes: a processing module 602 and a communication module 601. The processing module 602 is used to control and manage the actions of the embodied function implementation device 60 based on environmental constraint information. For example, it executes the steps of the display unit 61, the detection unit 62, the processing unit 63, and the sending unit 64, and / or is used to execute other processes of the technologies described herein. The communication module 601 is used for the interaction between the embodied function implementation device 60 based on environmental constraint information and other devices. As Figure 7 shown, the embodied function implementation device 60 based on environmental constraint information may further include a storage module 603, and the storage module 603 is used to store the program code and data of the embodied function implementation device 60 based on environmental constraint information.

[0073] Among them, the processing module 602 can be a processor or a controller. For example, it can be a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and so on. The communication module 601 can be a transceiver, an RF circuit, or a communication interface, etc. The storage module 603 can be a memory.

[0074] Among them, all relevant contents of each scenario involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated here. The above embodiment of the embodied function implementation device 60 based on environmental constraint information can execute the above Figure 2 shown method for implementing embodied functions based on environmental constraint information.

[0075] Please refer to Figure 8 , Figure 8 which is a schematic structural diagram of an electronic device proposed in an embodiment of this application. As Figure 8 shown, the electronic device 80 includes a processor 810, a memory 820, a communication interface 830, and one or more programs 821. The above one or more programs 821 are stored in the above memory and are configured to be executed by the above processor. When the program is executed, it includes some or all of the steps of any control method of the embodied robot described in the above method embodiments. The processor, the memory, and the communication interface are interconnected and complete the communication work between them.

[0076] Among them, the memory can be a volatile memory such as Dynamic Random Access Memory (DRAM), or a non-volatile memory such as a mechanical hard disk. The above memory is used to store a set of executable program codes. The above processor is used to call the executable program codes stored in the memory and can execute some or all of the steps of any method for implementing embodied functions based on environmental constraint information described in the above method embodiments of implementing embodied functions based on environmental constraint information.

[0077] It can be seen that for the electronic device 80 described in the embodiments of the present application, first, a function demonstration interface is displayed on the display screen of the user terminal, and the function demonstration interface includes a plurality of function demonstration controls; then, a trigger operation on a target function demonstration control among the plurality of function demonstration controls is detected, and the target function demonstration control is any one of the plurality of function demonstration controls determined by the user; after that, in response to the trigger operation, a scene detection request message is sent to the embodied robot; and, a first scene detection response message from the embodied robot is received, where the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt message, and the first prompt message is used to indicate that the environmental constraint information of the current target scene where the embodied robot is located can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint condition requirements and enhanced safety constraint condition requirements. The basic safety constraint condition requirements refer to that the unobstructed activity space of the embodied robot in the environmental constraint information meets the reference activity space required by the target function. The enhanced safety constraint condition requirements refer to that the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold. The sudden occupation of the task space means that the movable object occupies the reference activity space during the process of the embodied robot executing the target function; finally, a function demonstration instruction is sent to the embodied robot according to the first prompt message, and the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0078] After the user terminal of the present application receives a user instruction, compared with directly controlling the robot to execute the user instruction, this solution takes into account the influence of current environmental factors, sends a scene detection request to the robot, and analyzes through the robot whether there is an activity space corresponding to the target function in the current scene and the probability that the activity space may be occupied, so as to determine whether the target function can be demonstrated in the current environment. When the current environment can meet the requirements for demonstrating the target function, a function demonstration instruction is sent to the robot, enabling the robot to demonstrate the target function, which improves the safety and intelligence of the robot in executing user instructions.

[0079] The embodiments of the present application further provide a computer storage medium. The computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute some or all of the steps of any of the methods described in the above method embodiments. The above computer includes an electronic device.

[0080] The embodiments of the present application also provide a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any one of the methods described in the foregoing method embodiments. The computer program product may be a software installation package, and the computer includes an electronic device.

[0081] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0082] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0083] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, and the indirect coupling or communication connection of the device or unit may be in an electrical or other form.

[0084] The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software program module.

[0086] When the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, read-only memories (ROM), random access memories (RAM), external hard drives, magnetic disks, or optical discs.

[0087] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory, and the memory can include: flash drives, read-only memories, random access memories, magnetic disks, or optical discs, etc.

[0088] The above has introduced the embodiments of this application in detail. Specific examples are used in this article to elaborate on the principle and embodiments of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific embodiments and the scope of application. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A method for realizing embodied functions based on environmental constraint information, characterized in that: Applied to a user terminal, the user terminal is communicatively connected to an embodied robot, the method comprising: Displaying a function demonstration interface on a display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls; Detecting a triggering operation on a target function demonstration control among the multiple function demonstration controls, the target function demonstration control being any one of the multiple function demonstration controls determined by a user; In response to the trigger operation, a scene detection request message is sent to the embodied robot; and a first scene detection response message is received from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, and the first scene detection response message includes first prompt information, the first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function, the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements, the basic safety constraint requirements refer to the reference activity space required for the barrier-free activity space of the embodied robot in the environmental constraint information to meet the target function, the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function; A function demonstration instruction is sent to the embodied robot according to the first prompt information, wherein the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

2. The method according to claim 1, characterized in that The process of determining whether the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function includes the following steps: Obtaining environmental constraint information of a target scene in which the embodied robot is currently located; constructing a virtual scene corresponding to the target scene currently located by the embodied robot according to the environmental constraint information; Determining a target action corresponding to the target function, wherein the target action is used to indicate a technical action that the embodied robot needs to perform in order to complete the target function; Determining whether there is an interactive action in the target action, where the interactive action is an action in which the embodied robot interacts with the individual; If there is no interactive action in the target action, simulating the implementation process of the target action in the virtual scene to obtain image data when the embodied robot performs the target action in the virtual scene; Determine, based on the image data, whether the environmental constraint information can satisfy basic safety constraint condition requirements for demonstrating the target function.

3. The method according to claim 2, characterized in that The process of determining whether the environmental constraint information can satisfy the enhanced safety constraint condition requirement for demonstrating the target function comprises the following steps: Obtaining environmental constraint information of a target scene in which the embodied robot is currently located; Determining image information of each movable object in the target scene according to the environmental constraint information; Determine whether there is a space-affecting object according to the image information, wherein the space-affecting object is used to indicate an object whose behavior activity has uncertainty; If the space-affecting object does not exist, determining that the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function; If the space-influencing object exists, analyzing the constrained state of each space-influencing object to obtain the constrained state of each space-influencing object; Determine the probability of sudden occupation of the task space of each space influencing object according to the constraint condition of each space influencing object; Determine whether the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function based on the probability of sudden occupation of the task space of each space-influencing object and the preset probability threshold.

4. The method according to claim 1, characterized in that: The process of determining whether the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function includes the following steps: Obtaining environmental constraint information of a target scene in which the embodied robot is currently located; determining whether there is a target hidden danger according to the environmental constraint information, wherein the target hidden danger is used to indicate a condition, object or event that may potentially cause damage to the embodied robot when the embodied robot demonstrates the target function; If the target hidden danger exists, determining the correlation between each target hidden danger and a target action, wherein the target action is used to indicate the technical action that the embodied robot needs to perform in order to complete the target function; If the correlation between the target hidden danger and the target action is greater than a preset correlation, it is determined that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function; If there is no correlation between the target hidden danger and the target action greater than the preset correlation, it is determined that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

5. The method according to claim 2, characterized in that: The process of determining whether the environmental constraint information can satisfy the basic safety constraint condition requirement for demonstrating the target function also includes the following steps: If there is an interactive action in the target action, then determining the matching degree between each movable object in the environmental constraint information and the interactive action to obtain M matching degrees; Obtaining a matching degree greater than or equal to a preset matching degree among the M matching degrees, and obtaining N target matching degrees, where N is less than or equal to the M; If N is zero, it is determined that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function; If N is not zero, determining the movable object corresponding to each target matching degree in the N target matching degrees as a candidate interactive object, and obtaining N candidate interactive objects; Determining the action completion degree of each candidate interactive object and the embodied robot in the virtual scene to obtain N action completion degrees; When the action completion degree with the highest value among the N action completion degrees is greater than or equal to the preset completion degree, it is determined that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

6. The method according to claim 3, characterized in that After sending the scene detection request message to the embodied robot, the method further includes: receiving a second scene detection response message from the embodied robot, the second scene detection response message including second prompt information, the second prompt information being used to indicate that environmental constraint information of the target scene currently located by the embodied robot cannot satisfy the comprehensive condition requirement for demonstrating the target function; Sending an environment adjustment request message to the embodied robot according to the second prompt information; and receiving an environment adjustment response message from the embodied robot, wherein the adjustment response message includes third prompt information and a target environment adjustment scheme, wherein the third prompt information is used to indicate that a target result can be obtained by adjusting the environment constraint information through the target environment adjustment scheme, and the target result is used to indicate that the adjusted environment constraint information can meet the comprehensive condition requirements for demonstrating the target function; Sending an environment adjustment instruction to the embodied robot according to the third prompt information and the target environment adjustment scheme; and receiving an environment adjustment feedback message from the embodied robot; wherein the environment adjustment feedback message includes a fourth prompt information, and the fourth prompt information is used to indicate that the embodied robot has completed the adjustment of the environment constraint information according to the target environment adjustment scheme; The function demonstration instruction is sent to the embodied robot according to the fourth prompt information.

7. The method according to claim 6, characterized in that The process of determining the third prompt information and the target environment adjustment plan includes the following steps: Determine, based on the image data and / or the constraint conditions of each space-influencing object, why the environmental constraint information of the target scene currently located by the embodied robot cannot satisfy the comprehensive condition requirement for demonstrating the target function; determining at least one reference environment adjustment plan according to the cause; Determining the feasibility of each reference environment adjustment scheme in the at least one reference environment adjustment scheme; If there is a reference environment adjustment solution whose feasibility is greater than a preset feasibility, the reference environment adjustment solution with the highest feasibility is determined as the target environment adjustment solution; and the third prompt information is generated according to the target environment adjustment solution.

8. A device for realizing embodied functions based on environmental constraint information, characterized in that: Applied to a user terminal, the user terminal is communicatively connected with an embodied robot, and the device comprises: A display unit, configured to display a function demonstration interface on a display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls; A detection unit, configured to detect a triggering operation on a target function demonstration control among the multiple function demonstration controls, wherein the target function demonstration control is any one of the multiple function demonstration controls determined by a user; A processing unit, configured to respond to the trigger operation and send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, the first scene detection response message includes first prompt information, the first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function, the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements, the basic safety constraint requirements refer to the reference activity space required for the barrier-free activity space of the embodied robot in the environmental constraint information to meet the target function, the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information is less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function; A sending unit is used to send a function demonstration instruction to the embodied robot according to the first prompt information, and the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

9. An electronic device, characterized in that: It comprises a processor and a memory storing action execution instructions, wherein the memory stores one or more programs; when the processor executes the action execution instructions stored in the memory, the processor executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: An energy data management program is stored, including action execution instructions. When a processor of an electronic device executes the action execution instructions, the processor executes the method according to any one of claims 1-7.

Images