Method and related device for realizing embodied functions based on environmental constraint information

Abstract

The present application discloses an embodied function implementation method based on environmental constraint information and related devices, the method comprising: displaying a function demonstration interface on the display screen of the user terminal; detecting a trigger operation on a target function demonstration control among the multiple function demonstration controls; sending a scene detection request message to the embodied robot in response to the trigger operation; and receiving a first scene detection response message from the embodied robot, wherein the first scene detection response message includes a first prompt information, the first prompt information being used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function; and sending a function demonstration instruction to the embodied robot according to the first prompt information. The present application can improve the safety and intelligence of the embodied robot in executing user instructions.

Description

Technical Field

[0001] The present application relates to the field of robot control, and in particular to a method for realizing embodied functions based on environmental constraint information and related devices. Background Art

[0002] Currently, embodied robots are widely used in industrial manufacturing, home services, laboratory applications, service industries, and medical assistance. However, current embodied robots lack decision-making capabilities, which prevents them from accurately executing user commands. This can pose a threat to the environment and human safety, while also degrading the user experience. Summary of the Invention

[0003] The embodiments of the present application provide a method and related apparatus for realizing embodied functions based on environmental constraint information, so as to improve the safety and intelligence of an embodied robot in executing user commands.

[0004] In a first aspect, an embodiment of the present application provides a method for implementing an embodied function based on environmental constraint information, which is applied to a user terminal, wherein the user terminal is communicatively connected to an embodied robot, and the method includes:

[0005] Displaying a function demonstration interface on a display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls;

[0006] detecting a triggering operation on a target function demonstration control among the multiple function demonstration controls, the target function demonstration control being any one of the multiple function demonstration controls determined by a user;

[0007] In response to the trigger operation, a scene detection request message is sent to the embodied robot; and a first scene detection response message is received from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, and the first scene detection response message includes first prompt information, the first prompt information is used to indicate that the environmental constraint information of the target scene currently in which the embodied robot is located can meet the comprehensive condition requirements for demonstrating the target function, the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements, the basic safety constraint requirements refer to the reference activity space required for the unobstructed activity space of the embodied robot in the environmental constraint information to meet the target function, the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function;

[0008] A function demonstration instruction is sent to the embodied robot according to the first prompt information, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0009] The process of determining whether the environmental constraint information can satisfy the basic safety constraint requirements for demonstrating the target function includes the following steps:

[0010] Obtaining environmental constraint information of a target scene in which the embodied robot is currently located;

[0011] constructing a virtual scene corresponding to the target scene currently located by the embodied robot according to the environmental constraint information;

[0012] Determining a target action corresponding to the target function, where the target action is used to indicate a technical action that the embodied robot needs to perform in order to complete the target function;

[0013] Determining whether there is an interactive action in the target action, where the interactive action is an action in which the embodied robot interacts with the individual;

[0014] If there is no interactive action in the target action, simulating the implementation process of the target action in the virtual scene to obtain image data of the embodied robot performing the target action in the virtual scene;

[0015] Determine, based on the image data, whether the environmental constraint information can satisfy basic safety constraint requirements for demonstrating the target function.

[0016] The process of determining whether the environmental constraint information can satisfy the enhanced safety constraint requirement for demonstrating the target function includes the following steps:

[0017] Obtaining environmental constraint information of a target scene in which the embodied robot is currently located;

[0018] Determining image information of each movable object in the target scene according to the environmental constraint information;

[0019] determining whether there is a spatially influencing object according to the image information, wherein the spatially influencing object is used to indicate an object whose behavior activity is uncertain;

[0020] If the space-affecting object does not exist, determining that the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function;

[0021] If the spatial influencing object exists, analyzing the constrained state of each spatial influencing object to obtain the constrained state of each spatial influencing object;

[0022] Determining the probability of sudden occupation of the task space of each space-influencing object according to the constraint condition of each space-influencing object;

[0023] Determine whether the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function based on the probability of sudden occupation of the task space of each space-influencing object and the preset probability threshold.

[0024] The process of determining whether the environmental constraint information can satisfy the basic safety constraint requirements for demonstrating the target function includes the following steps:

[0025] Obtaining environmental constraint information of a target scene in which the embodied robot is currently located;

[0026] determining whether there is a target hidden danger based on the environmental constraint information, where the target hidden danger is used to indicate a condition, object, or event that may potentially cause damage to the embodied robot when the embodied robot demonstrates the target function;

[0027] If the target hidden danger exists, determining the degree of association between each target hidden danger and a target action, where the target action is used to indicate a technical action that the embodied robot needs to perform in order to complete the target function;

[0028] If the correlation between the target hidden danger and the target action is greater than a preset correlation, determining that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function;

[0029] If there is no correlation between the target hidden danger and the target action greater than the preset correlation, it is determined that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

[0030] The process of determining whether the environmental constraint information can satisfy the basic safety constraint requirements for demonstrating the target function further includes the following steps:

[0031] If there is an interactive action in the target action, determining the matching degree between each movable object in the environmental constraint information and the interactive action, and obtaining M matching degrees;

[0032] Obtaining matching degrees greater than or equal to a preset matching degree from the M matching degrees to obtain N target matching degrees, where N is less than or equal to the M;

[0033] If N is zero, it is determined that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function;

[0034] If N is not zero, determining the movable object corresponding to each target matching degree in the N target matching degrees as a candidate interactive object, to obtain N candidate interactive objects;

[0035] Determining the action completion degree of each candidate interactive object and the embodied robot in the virtual scene to obtain N action completion degrees;

[0036] When the action completion degree with the highest value among the N action completion degrees is greater than or equal to the preset completion degree, it is determined that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

[0037] After sending the scene detection request message to the embodied robot, the method further includes:

[0038] receiving a second scene detection response message from the embodied robot, the second scene detection response message including second prompt information, the second prompt information being used to indicate that environmental constraint information of a target scene currently located by the embodied robot cannot satisfy the comprehensive condition requirement for demonstrating the target function;

[0039] sending an environment adjustment request message to the embodied robot according to the second prompt information; and receiving an environment adjustment response message from the embodied robot, the adjustment response message including third prompt information and a target environment adjustment plan, the third prompt information being used to indicate that a target result can be obtained by adjusting the environmental constraint information through the target environment adjustment plan, and the target result being used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function;

[0040] sending an environment adjustment instruction to the embodied robot according to the third prompt information and the target environment adjustment plan; and receiving an environment adjustment feedback message from the embodied robot; wherein the environment adjustment feedback message includes a fourth prompt information, and the fourth prompt information is used to indicate that the embodied robot has completed the adjustment of the environmental constraint information according to the target environment adjustment plan;

[0041] The function demonstration instruction is sent to the embodied robot according to the fourth prompt information.

[0042] The process of determining the third prompt information and the target environment adjustment plan includes the following steps:

[0043] Determining, based on the image data and / or the constraint conditions of each of the spatial influencing objects, why the environmental constraint information of the target scene currently located by the embodied robot cannot satisfy the comprehensive condition requirements for demonstrating the target function;

[0044] determining at least one reference environment adjustment plan based on the cause;

[0045] determining the feasibility of each reference environment adjustment scheme in the at least one reference environment adjustment scheme;

[0046] If there is a reference environment adjustment solution whose feasibility is greater than the preset feasibility, the reference environment adjustment solution with the highest feasibility is determined as the target environment adjustment solution; and the third prompt information is generated according to the target environment adjustment solution.

[0047] In a second aspect, an embodiment of the present application provides an apparatus for implementing an embodied function based on environmental constraint information, which is applied to a user terminal, wherein the user terminal is communicatively connected to an embodied robot, and the apparatus includes:

[0048] A display unit, configured to display a function demonstration interface on a display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls;

[0049] a detection unit, configured to detect a triggering operation on a target function demonstration control among the plurality of function demonstration controls, wherein the target function demonstration control is any one of the plurality of function demonstration controls determined by a user;

[0050] a processing unit, configured to, in response to the trigger operation, send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot, wherein the scene detection request message includes a target function corresponding to the target function demonstration control, the scene detection request message is used to instruct the embodied robot to determine whether environmental constraint information of the target scene currently located can meet comprehensive condition requirements for demonstrating the target function, the first scene detection response message includes first prompt information, the first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet comprehensive condition requirements for demonstrating the target function, the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements, the basic safety constraint requirements refer to the reference activity space required for the unobstructed activity space of the embodied robot in the environmental constraint information to meet the target function, the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function;

[0051] A sending unit is used to send a function demonstration instruction to the embodied robot according to the first prompt information, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0052] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory storing action execution instructions, wherein the memory stores one or more programs; when the processor executes the action execution instructions stored in the memory, the processor executes the method described in the first aspect.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing an energy data management program, including action execution instructions. When a processor of an electronic device executes the action execution instructions, the processor executes the method described in the first aspect.

[0054] In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0055] It can be seen that in an embodiment of the present application, a function demonstration interface is first displayed on the display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls; then a triggering operation on a target function demonstration control among the plurality of function demonstration controls is detected, wherein the target function demonstration control is any one of the plurality of function demonstration controls determined by the user; thereafter, in response to the triggering operation, a scene detection request message is sent to the embodied robot; and a first scene detection response message is received from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, and the first scene detection response message includes a first prompt information, and the first prompt information The information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function, and the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements. The basic safety constraint requirements refer to the reference activity space required for the target function to be satisfied by the barrier-free activity space of the embodied robot in the environmental constraint information. The enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold. The sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function. Finally, a function demonstration instruction is sent to the embodied robot according to the first prompt information, and the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0056] After receiving the user instruction, the user terminal of this application takes into account the influence of the current environmental factors, compared with directly controlling the robot to execute the user instruction, and sends a scene detection request to the robot. The robot analyzes whether there is an activity space corresponding to the target function in the current scene and the probability that the activity space may be occupied, so as to determine whether the target function can be demonstrated in the current environment. If the current environment can meet the demonstration of the target function, the function demonstration instruction is sent to the robot, so that the robot demonstrates the target function, thereby improving the safety and intelligence of the robot in executing user instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0058] Figure 1 This is a system architecture diagram of a control system provided by an embodiment of the present application;

[0059] Figure 2 This is a flow chart of a method for implementing an embodied function based on environmental constraint information provided in an embodiment of the present application;

[0060] Figure 3 is a schematic diagram of an embodied robot selection interface provided in an embodiment of the present application;

[0061] Figure 4 This is a schematic diagram of a function demonstration interface provided in an embodiment of the present application;

[0062] Figure 5 is a schematic diagram of an outdoor scene provided in an embodiment of the present application;

[0063] Figure 6 This is a block diagram of the functional units of a device for realizing an embodied function based on environmental constraint information provided by an embodiment of the present application;

[0064] Figure 7 This is a block diagram of the functional units of another device for realizing an embodied function based on environmental constraint information provided by an embodiment of the present application;

[0065] Figure 8 This is a structural diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0068] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0069] Current embodied robots lack decision-making capabilities, which results in their inability to accurately execute user commands. This may pose a threat to the environment and personnel safety, while also reducing user experience.

[0070] In response to the above problems, an embodiment of the present application provides a method for realizing embodied functions based on environmental constraint information and a related device. The embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0071] See also Figure 1 , Figure 1 This is a system architecture diagram of a control system provided by an embodiment of the present application. Figure 1As shown, the control system 100 includes a user terminal 101 and a first embodied robot 102. The user terminal 101 can communicate with the first embodied robot 102 via a network or other connection method. The user terminal 101 is a device used by the user. The user can remotely control the embodied robot through an application program interface installed on the user terminal 101 to demonstrate functions, thereby generating function demonstration instructions. The generated function demonstration instructions are then sent from the user terminal 101 to the first embodied robot 102. The user terminal may include a tablet computer, a PDA, an in-vehicle electronic device, a server, a laptop computer, a mobile internet device (MID), a wearable electronic device (such as a smartwatch, a Bluetooth headset), a smartphone (such as an Android phone, an iOS phone, a Windows Phone phone, etc.). The above are merely examples, not an exhaustive list, and include but are not limited to the above electronic devices. The first embodied robot 102 analyzes the current embodied robot scene based on the received function demonstration instructions to determine whether the current scene is suitable for function demonstration and sends the result to the user terminal 101. Among them, the first embodied robot 102 embeds artificial intelligence into tangible entities such as robots, enabling them to have the ability to perceive, learn and dynamically participate in the surrounding environment. It can have a human-like appearance, including two feet, two arms and a head. It is a robot that imitates human functions and intelligence and can perform various tasks in human work and living environments.

[0072] See also Figure 2 , Figure 2 This is a flow chart of a method for implementing an embodied function based on environmental constraint information provided by an embodiment of the present application. The control method is applied to a user terminal that is in communication with an embodied robot. The method includes the following steps.

[0073] S210: Displaying a function demonstration interface on the display screen of the user terminal.

[0074] When a function demonstration request of the user for the embodied robot is detected, a function demonstration interface is displayed on the display screen, and the function demonstration interface includes a plurality of function demonstration controls.

[0075] Among them, the user terminal can be bound to multiple embodied robots, see Figure 3 , Figure 3 is a schematic diagram of an embodied robot selection interface provided in an embodiment of the present application, such as Figure 3As shown, before the user sends a function demonstration request, an embodied robot selection interface will be displayed on the display screen of the user terminal. The embodied robot selection interface includes multiple embodied robots bound to the user terminal, including the second embodied robot 1, the third embodied robot 2 and the fourth embodied robot 3, etc. The second embodied robot 1, the third embodied robot 2 and the fourth embodied robot 3 refer to the names of the embodied robots. There is configuration prompt information in the upper left corner of the embodied robot selection interface, which is used to prompt the user to select the embodied robot to be demonstrated. Next to the name of the embodied robot is the image data of the configured embodied robot for the user's reference. If the user clicks the control of the second embodied robot 1, an enlarged version of the image information of the embodied robot will be displayed on the right side of the interface. The user clicks the confirm control, and the user terminal enters the function demonstration interface. Please refer to Figure 4 , Figure 4 This is a schematic diagram of a function demonstration interface provided by an embodiment of the present application, such as Figure 4 As shown, the function demonstration interface includes multiple function demonstration controls, which may include shaking hands, squatting, bending over, somersaults, pouring water, taking out garbage, etc. The user can control the movement of the embodied robot through the direction control wheel 401 and the speed control wheel 402, and control the action of the embodied robot by clicking the skill demonstration control. The virtual space 403 is used to display the simulation results of the fifth embodied robot 404 according to the demonstration function selected by the user, and the simulation results are sent by the fifth embodied robot 404.

[0076] S220: Detecting a triggering operation on a target function demonstration control among the multiple function demonstration controls.

[0077] The target function demonstration control is any one of the multiple function demonstration controls determined by the user.

[0078] Among them, the user clicks on the demonstration function control, which corresponds to the trigger operation of the functional space. For example, if the user clicks on the squat control, the corresponding target function control is the squat control, indicating that the user needs to embody the robot to execute the squat instruction.

[0079] S230 , in response to the triggering operation, sending a scene detection request message to the embodied robot; and receiving a first scene detection response message from the embodied robot.

[0080] Among them, the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function. The first scene detection response message includes a first prompt information, and the first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function. The comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements. The basic safety constraint requirements refer to the reference activity space required for the target function to be satisfied by the barrier-free activity space of the embodied robot in the environmental constraint information. The enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold. The sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function.

[0081] After a user clicks a control for a demonstration function, the user terminal transmits the corresponding function to the embodied robot. Upon receiving the scene detection request, the embodied robot uses its camera to capture scene images from various angles and then analyzes the feasibility of the function demonstration based on the scene images. For example, if the user clicks a somersault control, the embodied robot determines whether the current environment is suitable for somersaults and whether it should execute the somersault instruction.

[0082] Specifically, the process of determining the first prompt information includes the following steps: the process of determining whether the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene in which the embodied robot is currently located; constructing a virtual scene corresponding to the target scene in which the embodied robot is currently located based on the environmental constraint information; determining the target action corresponding to the target function, the target action is used to indicate the technical action required for the embodied robot to perform in order to complete the target function; determining whether there is an interactive action in the target action, the interactive action is an action for the embodied robot to interact with an individual; if there is no interactive action in the target action, simulating the implementation process of the target action in the virtual scene to obtain image data when the embodied robot performs the target action in the virtual scene; determining whether the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function based on the image data.

[0083] Among them, the embodied robot obtains the current environmental information through the camera configured on the embodied robot. Furthermore, it can be video data from multiple angles. For example, when the embodied robot receives a scene detection request message, it can rotate in all directions to collect environmental information from various angles through the camera, thereby constructing a virtual scene, wherein the virtual scene is a three-dimensional graphical scene constructed based on the three-dimensional spatial information of the scene collected in real time.

[0084] Among them, the target function refers to the specific effect to be achieved, and the target action is the specific ability that needs to be mastered in order to achieve the effect. For example, if the target function is pouring water, the corresponding target action includes: finding a suitable container, placing the container in a stable position, or holding it in your hand so that it will not tip over or move during the pouring process, then picking up the container filled with water, making sure to hold the handle or neck of the container tightly to maintain stability and control, next, slowly tilting the container so that the water begins to flow out, adjusting the tilt angle of the container according to the amount of water to be poured out, and determining the water flow rate according to the capacity, and always observing the water level in the receiving container, when the water level reaches the required amount, stop pouring in time, after completing pouring, put the empty container back to its original position, make sure the container is placed firmly to avoid rolling or collision, if water splashes or overflows during the pouring process, it needs to be wiped clean with a paper towel or rag in time to keep the table or floor clean.

[0085] After the embodied robot determines the target action, it can also determine the category of the target action, that is, determine whether the target action is an interactive action or a separate demonstration action, and determine whether interaction with other individuals is required. If interaction is not required, the target action is determined to be a separate demonstration action. Then, the function is simulated according to the position of the embodied robot in the created virtual scene to obtain video data of the embodied robot completing the target function in the scene. Based on the video data, it is determined whether the current environment meets the basic safety constraint requirements, that is, whether the barrier-free activity space of the embodied robot meets the reference activity space required for the target function. The video data may show that the embodied robot can accurately perform the expected function, that is, the current environment meets the basic safety constraint requirements; it may also show that the embodied robot can perform part of the expected function, but deviations or errors occur in certain circumstances, and unexpected situations occur, that is, the current environment meets the basic safety constraint requirements. For example, there is an obstacle in the reference activity space and a collision occurs with the obstacle, or the embodied robot performs a somersault but there is a puddle in front, causing the embodied robot to fall.

[0086] It can be seen that in this embodiment, demonstrating the functions of the embodied robot in a virtual environment can avoid collisions, damage or adverse effects on the environment that may occur during actual execution, and can predict and identify potential errors and problems, which helps to ensure the safety and intelligence of the embodied robot operating in a real environment.

[0087] Specifically, the process of determining whether the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function also includes the following steps: if there is an interactive action in the target action, determining the matching degree between each movable object in the environmental constraint information and the interactive action to obtain M matching degrees; obtaining the matching degree greater than or equal to the preset matching degree among the M matching degrees to obtain N target matching degrees, where N is less than or equal to M; if N is zero, determining that the environmental constraint information cannot meet the basic safety constraint requirements for demonstrating the target function; if N is not zero, determining the movable object corresponding to each target matching degree in the N target matching degrees as a candidate interactive object to obtain N candidate interactive objects; determining the action completion degree of each candidate interactive object and the embodied robot in the virtual scene to obtain N action completion degrees; when the action completion degree with the highest value among the N action completion degrees is greater than or equal to the preset completion degree, determining that the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function.

[0088] Among them, if the category of the target action is determined to be an interactive action, it can be determined based on the scene screen whether there is an active object in the current scene. If there is no active object, a prompt message is sent to the user terminal to prompt the user that the interactive object does not exist and the target function cannot be demonstrated in the current scene.

[0089] If there are movable objects, multiple movable objects present in the current environmental constraint information are obtained, including children, young people, the elderly, pets, other robots, etc. Movable objects are used to indicate individuals that can move autonomously. Furthermore, the category of the interactive object, such as human, animal, or other robot, can be determined based on the interactive action. The multiple movable objects are filtered based on the determined category and interactive action. For example, if the interactive action is a handshake, animals and robots without arms can be filtered out to obtain multiple reference interactive objects.

[0090] The image of each reference interactive object is analyzed and identified to determine the degree of match between each reference interactive object and the interactive action. Specifically, the user's age, physical features, and features related to the interactive action can be identified. Physical features refer to extracting key point data of the human body from the image. These key points typically include joints, head, hands, and other locations. Features related to the interactive action include the human body posture and the object with which the user interacts. For example, if the user instruction is a handshake, the image of each interactive object is analyzed, focusing on the hands to determine whether a handshake operation is convenient. If the first interactive object is holding a child and the match is low, it is determined that a handshake operation is not possible. If the second interactive object is holding a drink in one hand and a shopping bag in the other, the match is also low, and it is determined that a handshake operation is not possible. If the third interactive object has both hands idle and the match is high, it is determined to be a candidate interactive object. If the fourth interactive object is holding a mobile phone in one hand and the other hand is idle, the match is relatively high and it is determined to be a candidate interactive object.

[0091] After obtaining the candidate interactive objects, the score of each candidate interactive object can be determined by indicators such as emotion recognition, physical feature analysis, location distance, etc., and each indicator is weighted and summed to obtain the score of each candidate interactive object. Then, based on the position of the candidate interactive object and the target action, the action simulation is carried out in the virtual scene to predict the completion of the target action. The target interactive object is determined based on the completion degree and the score of the candidate interactive object, and the completion degree is used to determine whether the current scene can achieve the target function. For example, the first candidate interactive object has a score of 89 points and a predicted completion of 90, the second candidate interactive object has a score of 93 points and a predicted completion of 87, the third candidate interactive object has a score of 91 and a predicted completion of 94. Assuming that the weight of the score is 0.3 and the weight of the predicted completion is 0.7, the total score of the first candidate interactive object is 89.7, the total score of the second candidate interactive object is 88.8, and the total score of the third candidate interactive object is 93.1. The third candidate interactive object is determined to be the target interactive object, and when the completion corresponding to the target interactive object is greater than or equal to the preset completion, it is determined that the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function. When the completion corresponding to the target interactive object is less than the preset completion, it is determined that the environmental constraint information cannot meet the basic safety constraint requirements for demonstrating the target function.

[0092] It can be seen that in the embodiments of the present application, by determining the interactive action, the interactive object, and predicting the completion degree of the action, the target can be located more accurately, thereby completing the task more efficiently.

[0093] Specifically, the process of determining whether the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene in which the embodied robot is currently located; determining whether there is a target hidden danger based on the environmental constraint information, and the target hidden danger is used to indicate the conditions, objects or events that may potentially cause damage to the embodied robot when the embodied robot demonstrates the target function; if the target hidden danger exists, determining the correlation between each target hidden danger and the target action, and the target action is used to indicate the technical action required for the embodied robot to complete the target function; if the correlation between the target hidden danger and the target action is greater than the preset correlation, determining that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function; if the correlation between the target hidden danger and the target action is greater than the preset correlation, determining that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

[0094] The embodied robot's range of motion can be determined based on the target action. The current environment can then be determined to determine whether the robot can move freely within this range. If so, the environment is then determined to contain potential hazards. These hazards can include obstacles in the scene, such as an unsecured kettle, plants, or charging cables, or hazards along the path, such as the flatness and friction of the ground. The correlation between multiple potential hazards and the target skill is then determined. If the correlation is greater than a preset correlation, the current environment is determined to be unsatisfactory for demonstrating the target skill's basic safety constraints. For example, if the potential hazards in the current environment include obstructions (plants) on the right and a puddle in front, and the target is a "somersault," the puddle has a high correlation with the target action, while the obstruction (plants) has a low correlation. Because the target potential hazard (the puddle) is highly correlated in the current environment, the environmental constraint information is determined to be unsatisfactory for demonstrating the target skill, and this result is transmitted to the device terminal.

[0095] It can be seen that in the embodiments of the present application, by determining whether there are dangerous factors in the current environment, the safety of the embodied robot can be improved, the task execution can be optimized, and the intelligence of the embodied robot in executing user instructions can be enhanced.

[0096] Specifically, the process of determining whether the environmental constraint information can meet the enhanced safety constraint requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene in which the embodied robot is currently located; determining the image information of each movable object in the target scene based on the environmental constraint information; determining whether there is a spatial influencing object based on the image information, and the spatial influencing object is used to indicate an object with uncertain behavioral activities; if the spatial influencing object does not exist, determining that the environmental constraint information can meet the enhanced safety constraint requirements for demonstrating the target function; if the spatial influencing object exists, analyzing the constrained state of each spatial influencing object to obtain the constrained situation of each spatial influencing object; determining the probability of sudden occupation of the task space of each spatial influencing object based on the constrained situation of each spatial influencing object; determining whether the environmental constraint information can meet the enhanced safety constraint requirements for demonstrating the target function based on the probability of sudden occupation of the task space of each spatial influencing object and the preset probability threshold.

[0097] Based on the image information of each movable object, the age range, category, and distance of each movable object from the reference activity space are determined. Multiple movable objects are then screened based on their age range, category, and distance to obtain spatially impacting objects. Specifically, these may include children under the age of 12, all pets, and humans whose distance from the reference activity space is less than a preset distance. The specific screening criteria are not limited herein. Next, the restraint status of each spatially impacting object is determined. For example, this may include whether a child is restrained, i.e., whether they are being led by their parents, or in a child-controllable or non-controllable vehicle. Alternatively, the restraint status of a pet may include whether it is on a leash and whether the pet's owner can restrain the pet. Specifically, the pet's breed can be identified, its personality determined, and whether the owner can restrain the pet based on its personality and the owner's body size. Based on the restraint status and the behavioral state of each spatially impacting object, the probability of sudden occupation of the task space for each spatially impacting object is determined. For example, a 10-year-old child who is not being held by his parents, is not crying or making a fuss, and is far away from the reference activity space, then the probability of sudden occupation of the task space of the space influencing object is low.

[0098] Specifically, image data of movable objects can be input into the interference probability model to output spatially influencing objects and the probability of sudden occupation of the task space by these spatially influencing objects. The interference probability model training process includes: first, collecting image data for each of the multiple movable objects. This can be achieved using sensors, cameras, or other monitoring devices. For example, cameras can be installed to monitor the position, behavior, and emotional state of each movable object. Sensors can also be used to detect other potential sources of interference in the environment, such as sound and light. Basic information about each movable object is then determined from the collected images, including information such as the age, gender, height, and weight of children; information such as the type, size, and personality of pets; and information such as the age, gender, height, and weight of pet owners; as well as their behavior patterns, range of activity, and interactions with other objects. Based on this basic information, spatially influencing objects are identified and features related to the restraint status of these spatially influencing objects are extracted. For example, whether a child is held by a parent or a pet is on a leash. The extracted features are then trained using machine learning or deep learning algorithms to establish the interference probability model. During training, historical or simulated data can be used as input, and model parameters can be continuously adjusted to optimize prediction performance. Expert knowledge or domain knowledge can also be considered to guide model training. In practical applications, new data must be continuously collected and the model updated to adapt to environmental changes. Furthermore, the model must be adjusted and optimized based on actual results to improve prediction accuracy. Through continuous iteration and improvement, the dynamic algorithm can more accurately predict spatially impacting objects and output the probability of sudden occupation of task space by these impacting objects.

[0099] Among them, if the probability of sudden occupation of the task space with a spatial influencing object is greater than or equal to a preset probability threshold, it is determined that the current environmental constraint information cannot meet the enhanced safety constraint requirements of the demonstration target function; if the probability of sudden occupation of the task space with no spatial influencing object is greater than or equal to the preset probability threshold, it is determined that the current environmental constraint information can meet the enhanced safety constraint requirements of the demonstration target function.

[0100] It can be seen that in this embodiment, the probability of sudden occupation of the task space is determined by the constraint status of the space influencing objects, and predictions are made for what may happen, so that potential problems can be discovered in time to protect the safety of on-site personnel and robots.

[0101] Specifically, after sending the scene detection request message to the embodied robot, the method further includes: receiving a second scene detection response message from the embodied robot, the second scene detection response message including second prompt information, the second prompt information being used to indicate that the environmental constraint information of the target scene currently located by the embodied robot cannot meet the comprehensive condition requirements for demonstrating the target function; sending an environment adjustment request message to the embodied robot according to the second prompt information; and receiving an environment adjustment response message from the embodied robot, the adjustment response message including third prompt information and a target environment adjustment scheme, the third prompt information being used to indicate that a target result can be obtained by adjusting the environmental constraint information through the target environment adjustment scheme, the target result being used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function; sending an environment adjustment instruction to the embodied robot according to the third prompt information and the target environment adjustment scheme; and receiving an environment adjustment feedback message from the embodied robot, wherein the environment adjustment feedback message includes fourth prompt information, the fourth prompt information being used to indicate that the embodied robot has completed adjustment of the environmental constraint information according to the target environment adjustment scheme; and sending the function demonstration instruction to the embodied robot according to the fourth prompt information.

[0102] Wherein, if the embodied robot determines that the environmental constraint information of the current target scene cannot meet the comprehensive condition requirements for demonstrating the target function and feeds it back to the user terminal, the user terminal sends an adjustment request to the embodied robot, and the embodied robot determines whether the current environment can be adjusted to achieve the target function. Further, the position or direction of the embodied robot, or the position of the obstacle, etc. can be adjusted. Based on the initial position and initial direction of the embodied robot, the simulated image data, and the determined target hidden danger, it is determined whether there is an adjustment plan. If an adjustment plan is obtained, the feasibility of each adjustment plan is determined, and the adjustment plan with the highest feasibility is determined as the target adjustment plan. The target adjustment plan is returned to the user terminal, so that the user terminal issues an adjustment instruction to the embodied robot according to the target adjustment plan, so that the embodied robot makes adjustments accordingly. After the adjustment is completed, the user terminal sends feedback information to the user terminal. After receiving the feedback information, the user terminal can first determine whether the adjustment of the embodied robot is accurate. If the adjustment is accurate, the function demonstration instruction is issued, so that the embodied robot demonstrates the target function in the real scene. If no adjustment plan is obtained, a message indicating that the current scene cannot achieve the target function is displayed on the function demonstration interface, along with the reason.

[0103] Specifically, the process of determining the third prompt information and the target environment adjustment plan includes the following steps: determining the reason why the environmental constraint information of the target scene in which the embodied robot is currently located cannot meet the comprehensive condition requirements for demonstrating the target function based on the image data and / or the constraint conditions of each spatial influencing object; determining at least one reference environment adjustment plan based on the reason; determining the feasibility of each reference environment adjustment plan in the at least one reference environment adjustment plan; if there is a reference environment adjustment plan whose feasibility is greater than a preset feasibility, determining the reference environment adjustment plan with the highest feasibility as the target environment adjustment plan; and generating the third prompt information based on the target environment adjustment plan.

[0104] The reason for non-satisfaction is determined based on the simulated video data and / or the constraints of each spatially influencing object. Specifically, if the basic safety constraint requirements are not met, the reason for non-satisfaction is determined based on the simulated video data or the correlation of the target hidden dangers; if the enhanced safety constraint requirements are not met, the reason for non-satisfaction is determined based on the constraints of each spatially influencing object. For example, if the basic safety constraint requirements are not met, performing a "somersault" task at the current position and direction and landing in a puddle could easily lead to unstable standing, or performing a "somersault" task at the current position and direction could cause the robot to collide with a table, causing it to fall or otherwise damage the embodied robot. It is then determined whether the reason for non-satisfaction is avoidable. If so, at least one adjustment solution is generated. For example, if there is a puddle in front of the embodied robot and the robot falls into it when performing a "somersault", the robot is then determined based on the current environment to determine whether it can move forward, backward, rotate, or move to another location. If so, a movement path is determined based on the landing location of the robot's "somersault". After moving to the corresponding location, the "somersault" task is then performed. For example, the first adjustment plan can be to rotate 30 degrees clockwise to avoid the direction of the puddle; the second adjustment plan can be to move forward 2 meters and cross the puddle. The feasibility of each adjustment plan is then calculated. If there is no feasibility greater than the preset feasibility, it is determined that there is no adjustment plan. If there is a feasibility greater than the preset feasibility, the adjustment plan corresponding to the highest feasibility is determined as the target adjustment plan and sent to the user terminal. For example, for the first adjustment plan, after rotating 30 degrees clockwise, a somersault task is performed. If there is a table obstacle, the feasibility is low. For the second adjustment plan, if the target action can be demonstrated safely by moving forward 2 meters and crossing the puddle to perform a somersault task, that is, the ground is stable and there are no obstacles, the feasibility is high and it is determined as the target adjustment plan.

[0105] It can be seen that in the embodiment of the present application, in the case where the current scene cannot demonstrate the target function, the target function is demonstrated by generating an adjustment plan, which increases the intelligence of the embodied robot when performing action demonstrations.

[0106] S240: Send a function demonstration instruction to the embodied robot according to the first prompt information.

[0107] The function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0108] After receiving the first prompt information from the embodied robot, the user terminal sends a function demonstration instruction to the embodied robot, causing the embodied robot to perform the function in a real-world scenario. For example, a user clicks on the embodied robot to demonstrate the "somersault" skill. If the embodied robot determines that the "somersault" skill is capable in the current scenario, the embodied robot performs the "somersault" task in the real-world scenario.

[0109] It can be seen that in the embodiment of the present application, after receiving the user instruction, the user terminal takes into account the influence of the current environmental factors compared to directly controlling the robot to execute the user instruction. The terminal sends a scene detection request to the robot, and the robot analyzes whether there is an activity space corresponding to the target function in the current scene and the probability that the activity space may be occupied, so as to determine whether the target function can be demonstrated in the current environment. When the current environment can meet the requirements for demonstrating the target function, a function demonstration instruction is sent to the robot, so that the robot demonstrates the target function, thereby improving the safety and intelligence of the robot in executing user instructions.

[0110] In one possible embodiment, see Figure 5 , Figure 5 is a schematic diagram of an outdoor scene provided in an embodiment of the present application, such as Figure 5As shown, the sixth embodied robot 506 is in an outdoor entertainment scene. The user selects the function that the current robot wants to demonstrate on the operation interface. When the user clicks the function control of playing table tennis, the function of playing table tennis is sent to the embodied robot, and the action corresponding to playing table tennis is determined. It is analyzed that playing table tennis is an interactive action, and the first interactive object 507, the second interactive object 508, and the third interactive object 509 are determined. At the same time, the current scene picture is obtained, and the target interactive object is determined among the first interactive object 507, the second interactive object 508, and the third interactive object 509 based on the scene picture. For example, it can be detected that the first interactive object 507 is young, the second interactive object 508 is difficult to move, and the third interactive object 509 is detected to have the highest match, and it is determined as the target interactive object. Furthermore, the sixth embodied robot 506 is controlled to move to the position of the target interactive object to invite the target interactive object to interact. After the target interactive object agrees, the table tennis playing in the current environment is simulated to determine that the stool 501 will hinder the execution of the action and that the first interactive object 507 is close to the table tennis table 505 and is prone to danger. The second prompt message is sent to the user terminal. After receiving the message, the user terminal sends an environment adjustment request message to the embodied robot. The embodied robot generates an adjustment plan based on the reason why the function cannot be realized. The adjustment plan may include prompting the first interactive object 507 to move to a farther The range and path of moving stool 501 are determined. For example, the path may be to move to the position of stool 501, lift stool 501, pass through first interactive object 507, and then place stool 501 around first lounge chair 502, second lounge chair 503, or third lounge chair 504. When passing through first interactive object 507, a prompt message is generated for first interactive object 507, informing it of the playing range and prompting first interactive object 507 to move to a safe position outside the playing range. The safe position may be the position of first lounge chair 502, second lounge chair 503, or third lounge chair 504. If the adjustment plan is determined to be feasible, it is sent to the user terminal, causing the user terminal to send an adjustment instruction to sixth embodied robot 506, causing the embodied robot to adjust and thereby demonstrate the function of playing table tennis with third interactive object 509.

[0111] For the same example as above, please refer to Figure 6 , Figure 6 This is a block diagram of the functional units of a device for realizing an embodied function based on environmental constraint information provided by an embodiment of the present application, such as Figure 6As shown, the embodied function implementation device 60 based on environmental constraint information includes: a display unit 61, used to display a function demonstration interface on the display screen of the user terminal, wherein the function demonstration interface includes multiple function demonstration controls; a detection unit 62, used to detect a triggering operation on a target function demonstration control among the multiple function demonstration controls, wherein the target function demonstration control is any one of the multiple function demonstration controls determined by the user; a processing unit 63, used to respond to the triggering operation and send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, and the first scene detection response message includes The embodiment includes a first prompt information, wherein the first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function, and the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements. The basic safety constraint requirements refer to the reference activity space required for the target function to be satisfied by the barrier-free activity space of the embodied robot in the environmental constraint information, and the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function; a sending unit 64 is used to send a function demonstration instruction to the embodied robot according to the first prompt information, and the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0112] Among them, the process of determining whether the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene in which the embodied robot is currently located; constructing a virtual scene corresponding to the target scene in which the embodied robot is currently located based on the environmental constraint information; determining the target action corresponding to the target function, and the target action is used to indicate the technical action required for the embodied robot to perform in order to complete the target function; determining whether there is an interactive action in the target action, and the interactive action is the action of the embodied robot to interact with the individual; if there is no interactive action in the target action, simulating the implementation process of the target action in the virtual scene to obtain image data when the embodied robot performs the target action in the virtual scene; and determining whether the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function based on the image data.

[0113] Among them, the process of determining whether the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene in which the embodied robot is currently located; determining the image information of each movable object in the target scene based on the environmental constraint information; determining whether there is a spatial influencing object based on the image information, and the spatial influencing object is used to indicate an object with uncertain behavioral activities; if the spatial influencing object does not exist, determining that the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function; if the spatial influencing object exists, analyzing the constrained state of each spatial influencing object to obtain the constrained situation of each spatial influencing object; determining the probability of sudden occupation of the task space of each spatial influencing object based on the constrained situation of each spatial influencing object; determining whether the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function based on the probability of sudden occupation of the task space of each spatial influencing object and the preset probability threshold.

[0114] Among them, the process of determining whether the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function includes the following steps: obtaining the environmental constraint information of the target scene in which the embodied robot is currently located; determining whether there is a target hidden danger based on the environmental constraint information, and the target hidden danger is used to indicate the conditions, objects or events that may potentially cause damage to the embodied robot when the embodied robot demonstrates the target function; if the target hidden danger exists, determining the correlation between each target hidden danger and the target action, and the target action is used to indicate the technical action required for the embodied robot to complete the target function; if the correlation between the target hidden danger and the target action is greater than the preset correlation, determining that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function; if the correlation between the target hidden danger and the target action is greater than the preset correlation, determining that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

[0115] Among them, the process of determining whether the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function also includes the following steps: if there is an interactive action in the target action, determining the matching degree between each movable object in the environmental constraint information and the interactive action to obtain M matching degrees; obtaining the matching degree greater than or equal to the preset matching degree among the M matching degrees to obtain N target matching degrees, where N is less than or equal to M; if N is zero, determining that the environmental constraint information cannot meet the basic safety constraint requirements for demonstrating the target function; if N is not zero, determining the movable object corresponding to each target matching degree in the N target matching degrees as a candidate interactive object to obtain N candidate interactive objects; determining the action completion degree of each candidate interactive object and the embodied robot in the virtual scene to obtain N action completion degrees; when the action completion degree with the highest value among the N action completion degrees is greater than or equal to the preset completion degree, determining that the environmental constraint information can meet the basic safety constraint requirements for demonstrating the target function.

[0116] In one possible embodiment, after sending the scene detection request message to the embodied robot, the embodied function implementation device 60 based on environmental constraint information is further specifically used to: receive a second scene detection response message from the embodied robot, the second scene detection response message including second prompt information, the second prompt information being used to indicate that the environmental constraint information of the target scene currently located by the embodied robot cannot meet the comprehensive condition requirements for demonstrating the target function; send an environment adjustment request message to the embodied robot according to the second prompt information; and receive an environment adjustment response message from the embodied robot, the adjustment response message including third prompt information and a target environment adjustment plan. The third prompt information is used to indicate that a target result can be obtained by adjusting the environmental constraint information through the target environment adjustment plan, and the target result is used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function; an environmental adjustment instruction is sent to the embodied robot according to the third prompt information and the target environment adjustment plan; and an environmental adjustment feedback message is received from the embodied robot; wherein the environmental adjustment feedback message includes a fourth prompt information, and the fourth prompt information is used to indicate that the embodied robot has completed the adjustment of the environmental constraint information according to the target environment adjustment plan; and the function demonstration instruction is sent to the embodied robot according to the fourth prompt information.

[0117] Among them, the process of determining the third prompt information and the target environment adjustment plan includes the following steps: determining the reason why the environmental constraint information of the target scene currently located by the embodied robot cannot meet the comprehensive condition requirements for demonstrating the target function based on the image data and / or the constraint conditions of each spatial influencing object; determining at least one reference environment adjustment plan based on the reason; determining the feasibility of each reference environment adjustment plan in the at least one reference environment adjustment plan; if there is a reference environment adjustment plan whose feasibility is greater than the preset feasibility, determining the reference environment adjustment plan with the highest feasibility as the target environment adjustment plan; and generating the third prompt information based on the target environment adjustment plan.

[0118] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0119] In the case of integrated units, see Figure 7 , Figure 7 This is a block diagram of the functional units of another embodiment of the present application of an embodiment of a device for realizing an embodied function based on environmental constraint information, such as Figure 7 As shown, the embodied function implementation device 60 based on environmental constraint information includes: a processing module 602 and a communication module 601. The processing module 602 is used to control and manage the actions of the embodied function implementation device 60 based on environmental constraint information, for example, executing the steps of the display unit 61, the detection unit 62, the processing unit 63 and the sending unit 64, and / or other processes for executing the technology described herein. The communication module 601 is used for the interaction between the embodied function implementation device 60 based on environmental constraint information and other devices. Figure 7 As shown, the apparatus 60 for realizing an embodied function based on environmental constraint information may further include a storage module 603 , which is used to store program codes and data of the apparatus 60 for realizing an embodied function based on environmental constraint information.

[0120] The processing module 602 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 601 may be a transceiver, an RF circuit, or a communication interface, and the like. The storage module 603 may be a memory.

[0121] Among them, all relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here. The above-mentioned embodied function implementation device 60 based on environmental constraint information can execute the above-mentioned Figure 2 The method for realizing embodied functions based on environmental constraint information is shown.

[0122] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device proposed in an embodiment of the present application. Figure 8 As shown, the electronic device 80 includes a processor 810, a memory 820, a communication interface 830 and one or more programs 821. The one or more programs 821 are stored in the memory and are configured to be executed by the processor. When the program is executed, it includes part or all of the steps of any one of the embodied robot control methods recorded in the above method embodiments. The processor, memory and communication interface are interconnected and complete communication with each other.

[0123] The memory can be a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a mechanical hard disk. The memory is used to store a set of executable program codes, and the processor is used to call the executable program codes stored in the memory and execute some or all of the steps of any of the methods for implementing embodied functions based on environmental constraint information as described in the embodiments of the method for implementing embodied functions based on environmental constraint information.

[0124] It can be seen that the electronic device 80 described in the embodiment of the present application first displays a function demonstration interface on the display screen of the user terminal, wherein the function demonstration interface includes multiple function demonstration controls; then detects a triggering operation on a target function demonstration control among the multiple function demonstration controls, wherein the target function demonstration control is any one of the multiple function demonstration controls determined by the user; thereafter, in response to the triggering operation, sends a scene detection request message to the embodied robot; and receives a first scene detection response message from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, and the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, and the first scene detection response message includes a first prompt information, The first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet the comprehensive condition requirements for demonstrating the target function, and the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements. The basic safety constraint requirements refer to the reference activity space required for the target function to be satisfied by the barrier-free activity space of the embodied robot in the environmental constraint information. The enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold. The sudden occupation of the task space refers to the occupation of the reference activity space by the movable object in the process of the embodied robot performing the target function. Finally, a function demonstration instruction is sent to the embodied robot according to the first prompt information, and the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

[0125] After receiving the user instruction, the user terminal of this application takes into account the influence of the current environmental factors, compared with directly controlling the robot to execute the user instruction, and sends a scene detection request to the robot. The robot analyzes whether there is an activity space corresponding to the target function in the current scene and the probability that the activity space may be occupied, so as to determine whether the target function can be demonstrated in the current environment. If the current environment can meet the demonstration of the target function, the function demonstration instruction is sent to the robot, so that the robot demonstrates the target function, thereby improving the safety and intelligence of the robot in executing user instructions.

[0126] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0127] The present application also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may comprise an electronic device.

[0128] It should be noted that for the aforementioned method implementations, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the implementations described in the specification are all optional implementations, and the actions and modules involved are not necessarily required for this application.

[0129] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0131] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0132] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0133] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: USB flash drives, read-only memories (ROM), random access memories (RAM), mobile hard drives, magnetic disks, or optical disks, and other media that can store program codes.

[0134] Those skilled in the art will understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0135] The above is a detailed introduction to the implementation methods of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above implementation methods is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for realizing embodied functions based on environmental constraint information, characterized in that: Applied to a user terminal, the user terminal being communicatively connected to an embodied robot, the method comprising: Displaying a function demonstration interface on a display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls; detecting a triggering operation on a target function demonstration control among the multiple function demonstration controls, the target function demonstration control being any one of the multiple function demonstration controls determined by a user; In response to the trigger operation, a scene detection request message is sent to the embodied robot; and a first scene detection response message is received from the embodied robot, wherein the scene detection request message includes the target function corresponding to the target function demonstration control, the scene detection request message is used to instruct the embodied robot to determine whether the environmental constraint information of the current target scene can meet the comprehensive condition requirements for demonstrating the target function, and the first scene detection response message includes first prompt information, the first prompt information is used to indicate that the environmental constraint information of the target scene currently in which the embodied robot is located can meet the comprehensive condition requirements for demonstrating the target function, the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements, the basic safety constraint requirements refer to the reference activity space required for the unobstructed activity space of the embodied robot in the environmental constraint information to meet the target function, the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function; A function demonstration instruction is sent to the embodied robot according to the first prompt information, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

2. The method according to claim 1, characterized in that The process of determining whether the environmental constraint information satisfies the basic safety constraint requirements for demonstrating the target function includes the following steps: Obtaining environmental constraint information of a target scene in which the embodied robot is currently located; constructing a virtual scene corresponding to the target scene currently located by the embodied robot according to the environmental constraint information; Determining a target action corresponding to the target function, where the target action is used to indicate a technical action that the embodied robot needs to perform in order to complete the target function; Determining whether there is an interactive action in the target action, where the interactive action is an action in which the embodied robot interacts with the individual; If there is no interactive action in the target action, simulating the implementation process of the target action in the virtual scene to obtain image data of the embodied robot performing the target action in the virtual scene; Determine, based on the image data, whether the environmental constraint information can satisfy basic safety constraint requirements for demonstrating the target function.

3. The method according to claim 2, characterized in that The process of determining whether the environmental constraint information satisfies the enhanced safety constraint requirement for demonstrating the target function comprises the following steps: Obtaining environmental constraint information of a target scene in which the embodied robot is currently located; Determining image information of each movable object in the target scene according to the environmental constraint information; determining whether there is a spatially influencing object according to the image information, wherein the spatially influencing object is used to indicate an object whose behavior activity is uncertain; If the space-affecting object does not exist, determining that the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function; If the spatial influencing object exists, analyzing the constrained state of each spatial influencing object to obtain the constrained state of each spatial influencing object; Determining the probability of sudden occupation of the task space of each space-influencing object according to the constraint condition of each space-influencing object; Determine whether the environmental constraint information can meet the enhanced safety constraint condition requirements for demonstrating the target function based on the probability of sudden occupation of the task space of each space-influencing object and the preset probability threshold.

4. The method according to claim 1, wherein The process of determining whether the environmental constraint information satisfies the basic safety constraint requirements for demonstrating the target function includes the following steps: Obtaining environmental constraint information of a target scene in which the embodied robot is currently located; determining whether there is a target hidden danger based on the environmental constraint information, where the target hidden danger is used to indicate a condition, object, or event that may potentially cause damage to the embodied robot when the embodied robot demonstrates the target function; If the target hidden danger exists, determining the degree of association between each target hidden danger and a target action, where the target action is used to indicate a technical action that the embodied robot needs to perform in order to complete the target function; If the correlation between the target hidden danger and the target action is greater than a preset correlation, determining that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function; If there is no correlation between the target hidden danger and the target action greater than the preset correlation, it is determined that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

5. The method according to claim 2, characterized in that The process of determining whether the environmental constraint information satisfies the basic safety constraint requirements for demonstrating the target function further includes the following steps: If there is an interactive action in the target action, determining the matching degree between each movable object in the environmental constraint information and the interactive action, and obtaining M matching degrees; Obtaining matching degrees greater than or equal to a preset matching degree from the M matching degrees to obtain N target matching degrees, where N is less than or equal to the M; If N is zero, it is determined that the environmental constraint information cannot meet the basic safety constraint condition requirements for demonstrating the target function; If N is not zero, determining the movable object corresponding to each target matching degree in the N target matching degrees as a candidate interactive object, to obtain N candidate interactive objects; Determining the action completion degree of each candidate interactive object and the embodied robot in the virtual scene to obtain N action completion degrees; When the action completion degree with the highest value among the N action completion degrees is greater than or equal to the preset completion degree, it is determined that the environmental constraint information can meet the basic safety constraint condition requirements for demonstrating the target function.

6. The method according to claim 3, characterized in that After sending the scene detection request message to the embodied robot, the method further includes: receiving a second scene detection response message from the embodied robot, the second scene detection response message including second prompt information, the second prompt information being used to indicate that environmental constraint information of a target scene currently located by the embodied robot cannot satisfy the comprehensive condition requirement for demonstrating the target function; sending an environment adjustment request message to the embodied robot according to the second prompt information; and receiving an environment adjustment response message from the embodied robot, the adjustment response message including third prompt information and a target environment adjustment plan, the third prompt information being used to indicate that a target result can be obtained by adjusting the environmental constraint information through the target environment adjustment plan, and the target result being used to indicate that the adjusted environmental constraint information can meet the comprehensive condition requirements for demonstrating the target function; sending an environment adjustment instruction to the embodied robot according to the third prompt information and the target environment adjustment plan; and receiving an environment adjustment feedback message from the embodied robot; wherein the environment adjustment feedback message includes a fourth prompt information, and the fourth prompt information is used to indicate that the embodied robot has completed the adjustment of the environmental constraint information according to the target environment adjustment plan; The function demonstration instruction is sent to the embodied robot according to the fourth prompt information.

7. The method according to claim 6, characterized in that The process of determining the third prompt information and the target environment adjustment plan includes the following steps: Determining, based on the image data and / or the constraint conditions of each of the spatial influencing objects, why the environmental constraint information of the target scene currently located by the embodied robot cannot satisfy the comprehensive condition requirements for demonstrating the target function; determining at least one reference environment adjustment plan based on the cause; determining the feasibility of each reference environment adjustment scheme in the at least one reference environment adjustment scheme; If there is a reference environment adjustment solution whose feasibility is greater than the preset feasibility, the reference environment adjustment solution with the highest feasibility is determined as the target environment adjustment solution; and the third prompt information is generated according to the target environment adjustment solution.

8. A device for realizing embodied functions based on environmental constraint information, characterized in that: Applied to a user terminal, the user terminal being communicatively connected to an embodied robot, the device comprising: A display unit, configured to display a function demonstration interface on a display screen of the user terminal, wherein the function demonstration interface includes a plurality of function demonstration controls; a detection unit, configured to detect a triggering operation on a target function demonstration control among the plurality of function demonstration controls, wherein the target function demonstration control is any one of the plurality of function demonstration controls determined by a user; a processing unit, configured to, in response to the trigger operation, send a scene detection request message to the embodied robot; and receive a first scene detection response message from the embodied robot, wherein the scene detection request message includes a target function corresponding to the target function demonstration control, the scene detection request message is used to instruct the embodied robot to determine whether environmental constraint information of the target scene currently located can meet comprehensive condition requirements for demonstrating the target function, the first scene detection response message includes first prompt information, the first prompt information is used to indicate that the environmental constraint information of the target scene currently located by the embodied robot can meet comprehensive condition requirements for demonstrating the target function, the comprehensive condition requirements include basic safety constraint requirements and enhanced safety constraint requirements, the basic safety constraint requirements refer to the reference activity space required for the unobstructed activity space of the embodied robot in the environmental constraint information to meet the target function, the enhanced safety constraint requirements refer to the probability of sudden occupation of the task space of the movable object in the environmental constraint information being less than a preset probability threshold, and the sudden occupation of the task space refers to the occupation of the reference activity space by the movable object during the process of the embodied robot performing the target function; A sending unit is used to send a function demonstration instruction to the embodied robot according to the first prompt information, where the function demonstration instruction is used to instruct the embodied robot to demonstrate the target function.

9. An electronic device, characterized in that: The method comprises a processor and a memory storing action execution instructions, wherein the memory stores one or more programs; when the processor executes the action execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that An energy data management program is stored, including action execution instructions. When a processor of an electronic device executes the action execution instructions, the processor executes the method according to any one of claims 1 to 7.

Images