Robot execution method, apparatus, electronic device, medium, and program product

Abstract

This disclosure discloses robot execution methods, apparatuses, electronic devices, media, and program products. One specific embodiment of the method includes: in response to acquiring real-time input information, determining a target control layer based on the real-time input information; generating first control processing information using the target control layer based on the real-time input information; generating robot processing information based on the first control processing information; and executing processing operations corresponding to the robot processing information. This embodiment relates to robot execution, and when the robot posture meets the control requirements corresponding to the input information, it can achieve continuous and natural changes in robot posture.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of robot motion control and emotional interaction technology, specifically to robot execution methods, devices, electronic devices, media, and program products. Background Technology

[0002] Currently, with the continuous development of intelligent robots, more and more robots are being applied in various scenarios to assist users in handling related events. How to achieve precise and natural control of robots has become one of the main research directions. For robot execution, the common approach is as follows: First, emotional or behavioral states such as "happy," "sad," and "neutral" are mapped to a set of preset posture parameters. Then, during state transitions, linear interpolation is used to transition from the current state posture to the target state posture, thus executing the posture change.

[0003] However, when using the above method, the following technical problems often arise: Different types of motion compete for the same control resources, which can lead to mutual overlap or cancellation. This may result in the main execution content being covered or canceled out, causing a mismatch between the content executed by the robot and the execution requirements. Summary of the Invention

[0004] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0005] Some embodiments of this disclosure provide robot execution methods, apparatuses, electronic devices, computer-readable media, and program products to address the technical problems mentioned in the background section above.

[0006] In a first aspect, some embodiments of this disclosure provide a robot execution method, including: in response to acquiring real-time input information, determining a target control layer based on the real-time input information; generating first control processing information using the target control layer based on the real-time input information; generating robot processing information based on the first control processing information; and performing processing operations corresponding to the robot processing information.

[0007] Optionally, the method further includes: in response to obtaining intermediate input information during the execution of the processing operation and determining that the target control layer is an interactive action control layer, obtaining the input time corresponding to the intermediate input information; obtaining robot processing sub-information after the input time and the robot execution state at the input time; generating second control processing information based on the intermediate input information and using the interactive action control layer; obtaining first weight change information corresponding to the second control processing information and second weight change information corresponding to the robot processing sub-information; using the robot execution state as the starting state, performing processing information fusion on the second control processing information and the robot processing sub-information based on the first weight change information and the second weight change information to obtain robot adjustment processing information; and executing the processing operation corresponding to the robot adjustment processing information.

[0008] Optionally, generating robot processing information based on the first control processing information includes: in response to the target control layer being an interactive action control layer, obtaining current robot processing information; obtaining third weight change information corresponding to the first control processing information and fourth weight change information corresponding to the current robot processing information, wherein the third weight change information is weight change information that gradually increases during the execution time period, and the fourth weight change information is weight change information that gradually decreases during the execution time period; and performing information fusion on the first control processing information and the current robot processing information based on the third weight change information and the fourth weight change information to obtain robot processing information.

[0009] Optionally, the method further includes: in response to the lack of input information in the robot's final state, acquiring future posture processing information for the posture control layer to perform posture processing in a future time period, wherein the robot's final state is the state after the processing operation is performed; acquiring fifth weight change information corresponding to the future posture processing information and sixth weight change information corresponding to the first control processing sub-information, wherein the first control processing sub-information is the control processing information in the future time period, the fifth weight change information is the weight change information that gradually increases in the future time period, and the sixth weight change information is the weight change information that gradually decreases in the future time period; taking the robot's final state as the starting state, performing processing information fusion on the future posture processing information and the first control processing sub-information based on the fifth weight change information and the sixth weight change information to obtain first future processing information; and performing the processing operation corresponding to the first future processing information.

[0010] Optionally, the above method further includes: in response to obtaining micro-expression control information at a target future time, obtaining micro-expression processing information output by the micro-expression control layer, the robot's future state at the target future time, and future processing sub-information; obtaining the seventh weight change information corresponding to the future processing sub-information and the eighth weight change information corresponding to the micro-expression processing information, wherein the seventh weight change information is a weight change information that gradually decreases during the execution time period, and the eighth weight change information is a weight change information that gradually increases and then gradually decreases during the execution time period; taking the robot's future state as the starting state, performing processing information fusion on the future processing sub-information and the micro-expression processing information based on the seventh weight change information and the eighth weight change information to obtain second future processing information; and performing the processing operation corresponding to the second future processing information at the target future time.

[0011] Optionally, the aforementioned target control layer is a control layer in the control layer set used for superimposed control that focuses on processing the aforementioned real-time input information. The control layer set includes: a posture control layer based on overall emotion, an interactive action control layer, and a micro-expression control layer. Each control layer in the control layer set is a control layer that adjusts the robot's state at different time scales. The posture control layer is a minute-level posture control layer, the interactive action control layer is a second-level action interaction processing layer, and the micro-expression control layer is a millisecond-level micro-expression processing layer.

[0012] Optionally, the above-mentioned generation of first control processing information based on the real-time input information and using the target control layer includes: generating personalized processing feature information based on the object profile of the interactive object, wherein the personalized processing feature information includes at least one of processing duration, processing range, processing speed, and processing style; and generating the first control processing information based on the personalized processing feature information and the real-time input information and using the target control layer.

[0013] Optionally, the aforementioned target control layer is a posture control layer; and the generation of first control processing information using the aforementioned target control layer based on the aforementioned personalized processing feature information and the aforementioned real-time input information includes: generating facial expression processing information using the aforementioned posture control layer based on the aforementioned processing speed, the aforementioned processing style and the aforementioned real-time input information, as the first control processing information.

[0014] Optionally, the trigger threshold corresponding to the micro-expression control layer is determined based on the object sensitivity of the interactive object; and the acquisition of the seventh weight change information corresponding to the future processing sub-information and the eighth weight change information corresponding to the micro-expression processing information includes: setting the weight decay information corresponding to the micro-expression processing information according to the object sensitivity; and setting the eighth weight change information corresponding to the micro-expression processing information according to the weight decay information.

[0015] Optionally, obtaining the fifth weight change information corresponding to the future attitude processing information and the sixth weight change information corresponding to the first control processing sub-information includes: determining the duration of the weight change based on the execution environment corresponding to the future time period; and setting the fifth weight change information and the sixth weight change information based on the duration of the weight change.

[0016] Optionally, obtaining the third weight change information corresponding to the first control processing information and the fourth weight change information corresponding to the current robot processing information includes: obtaining the corresponding weight selection strategy based on the execution environment within the execution time period; and setting the third weight change information and the fourth weight change information based on the weight selection strategy.

[0017] Secondly, some embodiments of this disclosure provide a robot execution device, including: a determining unit configured to determine a target control layer based on real-time input information in response to acquiring real-time input information; a first generating unit configured to generate first control processing information using the target control layer based on the real-time input information; a second generating unit configured to generate robot processing information based on the first control processing information; and an execution unit configured to execute processing operations corresponding to the robot processing information.

[0018] Optionally, the apparatus further includes: in response to obtaining intermediate input information during the execution of a processing operation and determining that the target control layer is an interactive action control layer, obtaining the input time corresponding to the intermediate input information; obtaining robot processing sub-information after the input time and the robot execution state at the input time; generating second control processing information based on the intermediate input information and using the interactive action control layer; obtaining first weight change information corresponding to the second control processing information and second weight change information corresponding to the robot processing sub-information; using the robot execution state as the starting state, performing processing information fusion on the second control processing information and the robot processing sub-information based on the first weight change information and the second weight change information to obtain robot adjustment processing information; and executing the processing operation corresponding to the robot adjustment processing information.

[0019] Optionally, the first generation unit can be configured to: in response to the target control layer being an interactive action control layer, acquire current robot processing information; acquire third weight change information corresponding to the first control processing information and fourth weight change information corresponding to the current robot processing information, wherein the third weight change information is a weight change information that gradually increases during the execution time period, and the fourth weight change information is a weight change information that gradually decreases during the execution time period; and perform processing information fusion on the first control processing information and the current robot processing information based on the third weight change information and the fourth weight change information to obtain robot processing information.

[0020] Optionally, the apparatus further includes: in response to no input information being acquired in the robot's final state, acquiring future attitude processing information for attitude processing by the attitude control layer in a future time period, wherein the robot's final state is the state after the processing operation is performed; acquiring fifth weight change information corresponding to the future attitude processing information and sixth weight change information corresponding to the first control processing sub-information, wherein the first control processing sub-information is control processing information in the future time period, the fifth weight change information is weight change information that gradually increases in the future time period, and the sixth weight change information is weight change information that gradually decreases in the future time period; taking the robot's final state as the starting state, performing processing information fusion on the future attitude processing information and the first control processing sub-information based on the fifth weight change information and the sixth weight change information to obtain first future processing information; and performing the processing operation corresponding to the first future processing information.

[0021] Optionally, the device further includes: in response to acquiring micro-expression control information at a target future time, acquiring micro-expression processing information output by the micro-expression control layer, the robot's future state at the target future time, and future processing sub-information; acquiring seventh weight change information corresponding to the future processing sub-information and eighth weight change information corresponding to the micro-expression processing information, wherein the seventh weight change information is a weight change information that gradually decreases during the execution time period, and the eighth weight change information is a weight change information that gradually increases and then gradually decreases during the execution time period; taking the robot's future state as the starting state, performing processing information fusion on the future processing sub-information and the micro-expression processing information based on the seventh weight change information and the eighth weight change information to obtain second future processing information; and performing the processing operation corresponding to the second future processing information at the target future time.

[0022] Optionally, the aforementioned target control layer is a control layer in the control layer set used for superimposed control that focuses on processing the aforementioned real-time input information. The control layer set includes: a posture control layer based on overall emotion, an interactive action control layer, and a micro-expression control layer. Each control layer in the control layer set is a control layer that adjusts the robot's state at different time scales. The posture control layer is a minute-level posture control layer, the interactive action control layer is a second-level action interaction processing layer, and the micro-expression control layer is a millisecond-level micro-expression processing layer.

[0023] Optionally, the first generation unit can be configured to: generate personalized processing feature information based on the object profile of the interactive object, wherein the personalized processing feature information includes at least one of processing duration, processing range, processing speed, and processing style; and generate first control processing information based on the personalized processing feature information and the real-time input information, using the target control layer.

[0024] Optionally, the target control layer is an attitude control layer; and the first generation unit can be configured to generate facial expression processing information as first control processing information based on the processing speed, the processing style and the real-time input information, using the attitude control layer.

[0025] Optionally, the trigger threshold corresponding to the micro-expression control layer is determined based on the object sensitivity of the interactive object; and the first generation unit can be configured to: set the weight decay information corresponding to the micro-expression processing information according to the object sensitivity; and set the eighth weight change information corresponding to the micro-expression processing information according to the weight decay information.

[0026] Optionally, the first generation unit can be configured to: determine the duration of weight change based on the execution environment corresponding to the aforementioned future time period; and set the fifth weight change information and the sixth weight change information based on the aforementioned duration of weight change.

[0027] Optionally, the first generation unit can be configured to: obtain the corresponding weight selection strategy based on the execution environment within the aforementioned execution time period; and set the aforementioned third weight change information and the aforementioned fourth weight change information based on the aforementioned weight selection strategy.

[0028] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0029] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0030] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0031] The various embodiments of this disclosure have the following beneficial effects: Through the robot execution methods of some embodiments of this disclosure, continuous and natural changes in robot posture can be achieved when the robot posture meets the control requirements corresponding to the input information. Specifically, the reason why the changes in robot posture are not natural enough and do not meet the semantic execution requirements is that different types of motion compete for the same control resources, resulting in mutual overriding or cancellation. This may lead to the main execution content being overridden or cancelled, causing the content executed by the robot to mismatch with the execution requirements. Based on this, the robot execution method of some embodiments of this disclosure firstly, in response to the target robot obtaining real-time input information, determines the target control layer based on the real-time input information. Here, by setting a posture control layer, an interactive action control layer, and a micro-expression control layer, the control layer that needs to output the main content can be accurately located based on the different real-time input information, so as to achieve adaptive posture changes for different types of input information (i.e., different types of event processing requests). In addition, by determining the target control layer, the generation of the main execution semantic content can be realized, ensuring that the main execution semantic content is not overridden or cancelled during the subsequent execution of the target robot, thus ensuring the correctness of the target robot's corresponding execution semantics. Then, based on the aforementioned real-time input information, the target control layer generates first control processing information for use in generating the semantic content to be actually executed by the robot. Next, robot processing information is generated based on the first control processing information. Here, under the constraints of the first control processing information, and while ensuring that the semantic content of the execution does not deviate from the execution content corresponding to the first control processing information, the robot processing information can be accurately generated. Finally, the processing operation corresponding to the robot processing information is executed. In summary, by determining the target control layer based on real-time input information and constraining the semantic content of the first control processing information output by the target control layer, the robot's posture can be continuously and naturally changed during execution, ensuring that the execution content does not deviate from the execution requirements corresponding to the real-time input information. Attached Figure Description

[0032] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0033] Figure 1 This is a schematic diagram illustrating an application scenario of a robot execution method according to some embodiments of the present disclosure; Figure 2 This is a flowchart of some embodiments of the robot execution method according to the present disclosure; Figure 3 The diagram illustrates the weight changes corresponding to the posture control layer, interaction action control layer, and micro-expression control layer. Figure 4 These are flowcharts of other embodiments of the robot execution method according to this disclosure; Figure 5 These are schematic diagrams illustrating the structure of some embodiments of the robot actuator according to this disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0034] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0035] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0036] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0037] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0038] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0039] Before performing any of the operations involving the collection, storage, or use of user personal information (e.g., real-time input information) disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.

[0040] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] Figure 1 This is a schematic diagram of an application scenario of a robot execution method according to some embodiments of the present disclosure.

[0042] exist Figure 1 In this application scenario, firstly, in response to receiving real-time input information 102, the target robot 101 can determine the target control layer 103 based on the real-time input information 102. In this application scenario, the real-time input information 102 can be "information about touching the robot's head". Other control layers also exist. That is, the control layers corresponding to the target robot 101 can include: a first control layer, a target control layer, and a second control layer. Then, the target robot 101 can generate first control processing information 104 based on the real-time input information 102 and the target control layer 103. In this application scenario, the first control processing information 104 can be a joint parameter sequence. Next, the target robot 101 can generate robot processing information 105 based on the first control processing information 104. In this application scenario, the robot processing information 105 can be a joint parameter sequence of a shy action of "looking down + covering face with hands". Finally, the target robot 101 can execute the processing operation corresponding to the robot processing information 105.

[0043] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of a robot execution method according to the present disclosure. The robot execution method includes the following steps: Step 201: In response to obtaining real-time input information, determine the target control layer based on the real-time input information.

[0044] In some embodiments, in response to the target robot acquiring real-time input information, the execution subject of the robot execution method (e.g., based on the real-time input information) is... Figure 1The target robot 101 shown can determine the target control layer based on the aforementioned real-time input information. The target robot can be the robot currently interacting with or awaiting the execution of relevant instructions. In practice, the target robot can be various types of robots. For example, the target robot can be a companion robot, service robot, desktop robot, or an intelligent robot with emotional interaction capabilities. The real-time input information can be the input information related to posture control that the target robot acquires in real time. In practice, the real-time input information can be event information that requires the target robot to provide event feedback on relevant events. For example, the real-time input information can include, but is not limited to, at least one of the following: voice event information, touch event information, posture recognition results, visual / acoustic environmental event information, and emotion engine output results. The aforementioned target control layer is a control layer that focuses on processing the aforementioned real-time input information. That is, it generates a control layer based on the real-time input information to assist the target robot in performing relevant processing operations. The target control layer can be at least one control layer selected based on the different event types corresponding to the real-time input information, primarily responsible for outputting posture changes. That is, the target control layer can be one control layer or multiple control layers. In other words, depending on the event information corresponding to the real-time input information, there may be multiple control layers or only one layer responsible for outputting the main posture changes. The specific layer is determined primarily by the event type. In practice, the target robot has multiple control layers that work together to execute the commands. The target control layer can be a control layer related to the semantic content type corresponding to the real-time input information. For example, multiple control layers could be control layers for various robot joints within the target robot. Based on the joint information involved in the real-time input information, the corresponding associated control layer can be selected as the target control layer to achieve the transformation of semantic content into relevant joint control information.

[0045] As an example, firstly, the real-time input information can be semantically parsed to determine the content type. Then, based on the mapping relationship between the content type and the control layer, the control layer corresponding to the real-time input information can be determined as the target control layer.

[0046] In some optional implementations of certain embodiments, the aforementioned target control layer is a control layer within a set of control layers used for superimposed control that focuses on processing the aforementioned real-time input information. In practice, for real-time input information that primarily affects the robot's corresponding emotions, the corresponding target control layer could be a posture control layer. For example, for real-time input information that primarily instructs the robot to perform related actions, the corresponding target control layer could be an interactive action control layer. For real-time input information that primarily instructs the robot to perform micro-expression changes, the corresponding target control layer could be a micro-expression control layer. For yet another example, for real-time input information that primarily instructs the robot to perform related actions with emotional variations, the corresponding target control layer could be both an interactive action control layer and a posture control layer. The set of control layers for superimposed control can be achieved by superimposing the processed information from each control layer's output to control the target robot's posture changes. A posture control layer based on overall emotion can be a control layer used to maintain the robot's emotional tone and posture over a longer time period. The posture control layer has a low parameter update rate, and its output does not pursue strong movements but rather simulates the overall posture drift caused by changes in human emotional tone. For example, a slight tilt of the head, a gradual expansion of the shoulders, or a slow forward lean of the torso. In specific scenarios, the posture control layer can be software logic and an AI model. The software logic corresponding to the posture control layer typically uses a state machine or behavior tree. The AI ​​model can be an emotion computing model and procedural animation generation. The emotion computing model can analyze the user's semantics and speech based on a large language model (LLM) or a multimodal model, outputting emotion labels (e.g., a PAD emotion model) to determine the current basic posture. Procedural animation generation can combine a simple physics engine or inverse kinematics (IK) to fine-tune the skeletal posture based on emotion parameters, making the idle movements appear natural and fluid. For example, the software logic corresponding to the interactive action control layer is an animation blending and action queue management. The corresponding AI model can be a Vision-Language-Action (VLA) model and reinforcement learning (RL) with action primitives. The software logic corresponding to the micro-expression control layer involves performing high-frequency numerical perturbations on the deformation targets (blendshapes) or local joints of the facial mesh. The AI ​​model corresponding to the micro-expression control layer can be a temporal prediction model (LSTM / RNN) and temporal smoothing and optical flow constraints.

[0047] The structural representation of the robot processing information output by the attitude control layer can be: L1_state = { emotion_type, joint_params_base[1...n], transition_speed, intensity }

[0048] Here, L1 can be the attitude control layer. `emotion_type` represents the emotion type, `joint_params_base` represents the set of basic joint parameters, `ransition_speed` represents the basic attitude drift rate, and `intensity` represents the current emotion intensity.

[0049] An interaction action control layer is a control layer used to execute specific interactive actions. It responds to specific interactive events, such as nodding, clapping, shaking one's head, tilting one's head, raising one's hand, covering one's face, moving closer, or leaning back.

[0050] The structural representation of the robot processing information output by the interactive action control layer can be: L2_state = { current_action, joint_params_current[1...n], joint_params_target[1...n], progress, last_frame[1...n], interrupted }

[0051] Here, L2 can be the interactive action control layer. `joint_params_current` represents the actual joint values ​​at the current moment. `joint_params_target` represents the target action parameters. `progress` represents the action progress. `last_frame` represents the parameters persisted in the last frame.

[0052] The micro-expression control layer can be used to superimpose high-frequency detail perturbations onto the outputs of the posture control layer and the interaction action control layer. The micro-expression control layer can provide anthropomorphic performance at the level of detail. For example, anthropomorphic performance may include, but is not limited to: slight head retraction after being touched, slight torso undulation due to breathing, brief tremor upon being startled or called, gaze shift or subtle head turn, slight shaking, and touch feedback. The output of the micro-expression control layer is superimposed on the synthesized result of the posture control layer and the interaction action control layer, without modifying the state of the posture control layer and the interaction action control layer themselves. In specific scenarios, In a specific example, the aforementioned execution entity uses a pre-set matching strategy to determine the target control layer corresponding to the real-time input information. For example, the pre-set matching strategy could be the following rules: (1) Changes in the state of the emotion engine drive the slow update of the L1 layer.

[0053] (2) The L2 layer action is triggered when the voice interaction is completed, the task is confirmed, or the gesture recognition result is obtained.

[0054] (3) Light touch, short-term tap, instantaneous acoustic stimulation, etc., trigger instantaneous micro-response in L3 layer.

[0055] (4) Long press, continuous touch or events that require a clear response can trigger L2 layer actions.

[0056] (5) Safety incidents, collision risks, and joint protection incidents can bypass the L1 / L2 / L3 layers and directly issue constraint or braking commands to the execution layer.

[0057] Optionally, for the control layer set, not only is parallelism between control layers supported, but also differentiated processing of events with different priorities is supported.

[0058] In addition, the various control layers in the aforementioned control layer cluster are control layers that adjust the robot's state at different time scales. That is, the output content of each control layer is robot processing information output for different time scales. Here, through robot processing at different time scales, the processing information at different time scales can be superimposed and processed to achieve the natural execution of complex actions. The aforementioned posture control layer is a minute-level posture control layer. That is, the robot processing information output by the posture control layer is posture change information at a minute level or slower. The aforementioned interactive action control layer is a second-level action interaction processing layer. Therefore, the robot processing information output by the interactive action control layer is second-level action expression information. The aforementioned micro-expression control layer is a millisecond-level micro-expression processing layer. Among them, the robot processing information output by the micro-expression control layer is millisecond-level or high-frequency short-term posture perturbation information. The robot processing information output by the micro-expression control layer has the characteristics of fast triggering, small amplitude, and short duration.

[0059] Step 202: Based on the real-time input information mentioned above, generate first control processing information using the target control layer mentioned above.

[0060] In some embodiments, the execution entity can generate first control processing information based on the real-time input information and the target control layer. The first control processing information can be candidate processing content for the pose processing of the target robot. In practice, the first control processing information can be processing content in the form of joint parameters. That is, the first control processing information is the change information of each joint parameter corresponding to each joint. Specifically, the first control processing information can be a sequence of joint parameters indicating the robot's joints over a period of time. Each joint has a corresponding joint parameter sequence. The joint parameter sequence can characterize the change of joint parameter values ​​over a period of time.

[0061] As an example, the aforementioned execution entity can input real-time input information to the target control layer to obtain the first control processing information.

[0062] In some optional implementations of certain embodiments, the execution entity can generate first control processing information based on the real-time input information and the target control layer, including the following steps: The first step is to generate personalized processing feature information based on the object profile of the interaction object. This personalized processing feature information includes at least one of processing duration, processing amplitude, processing speed, and processing style. The interaction object can be an object that interacts with the target robot (e.g., a user). For example, if the target robot is a companion robot, the corresponding interaction object is a companion object. The object profile can be a profile describing the behavioral characteristics of the corresponding object. The personalized processing feature information can be processing feature content on how the robot processes information based on the personalized behavioral characteristics and performance of the interaction object. That is, under the constraints of the personalized processing feature information, the robot's posture changes can conform to the posture behavior changes preferred by the interaction object. Processing duration can be the duration of the robot's posture changes in response to real-time input information. Processing amplitude can be the amplitude of the robot's corresponding posture changes. Processing speed can be the speed of the robot's posture changes. Processing style can be the style of the robot's corresponding posture changes.

[0063] As an example, the aforementioned execution entity can determine the processing features for real-time input information under the object profile by querying historical posture change records and association mapping tables, thereby obtaining personalized processing feature information. Historical posture change records can be records of posture changes in response to input event information over historical time periods. The association mapping table can represent the mapping relationship between event information and event processing feature requirements. The association mapping table can be generated based on an offline multimodal large model. Event information can be the event content corresponding to the real-time input information. Event processing feature requirements can be the feature content of the robot characteristics required when processing the event.

[0064] The second step involves generating first control processing information based on the aforementioned personalized processing feature information and real-time input information, using the aforementioned target control layer.

[0065] As an example, the aforementioned execution entity can input the aforementioned personalized processing feature information and the aforementioned real-time input information into the target control layer, so as to output the first control processing information according to the various feature requirements corresponding to the aforementioned personalized processing feature information.

[0066] Optionally, the aforementioned target control layer is an attitude control layer. Here, when the target control layer is determined to be an attitude control layer, the real-time input information is characterized as the input information that influences the overall emotional change of the target robot. For example, the real-time input information is "information requiring the robot to express an overall happy emotion".

[0067] Optionally, the generation of first control processing information based on the personalized processing feature information and the real-time input information, using the target control layer, includes the following steps: The aforementioned execution entity can generate facial expression processing information, as the first control processing information, based on the aforementioned processing speed, processing style, and real-time input information, using the aforementioned posture control layer. This facial expression processing information can be joint parameter changes that require the target robot to perform posture changes so that the robot's overall emotional expression reflects the emotional content associated with the real-time input information.

[0068] As an example, the aforementioned execution entity can input the aforementioned processing speed, processing style, and real-time input information into the posture control layer to output facial expression processing information according to the processing speed and processing style.

[0069] Step 203: Generate robot processing information based on the first control processing information mentioned above.

[0070] In some embodiments, the aforementioned execution entity can generate robot processing information based on the first control processing information. The robot processing information can be information about how the robot will actually execute and process subsequent operations. It should be noted that the robot processing information here can also be the joint parameter sequence corresponding to each joint. Through the robot processing information, the target robot can perform the corresponding processing operations to respond to the semantic control requirements corresponding to the aforementioned real-time input information.

[0071] It should be noted that the information processed by the robot can be the fused information from the various control processing information output by each control layer (including the target control layer).

[0072] As an example, firstly, the control processing information corresponding to each control layer set other than the target control layer is obtained. Then, the weight of the first control processing information is set higher than the weight of each of the other control processing information. Next, based on the weight of the first control processing information and the weights of the other information, the first control processing information and the other control processing information are fused together using corresponding joint parameters to obtain the joint superposition parameters for each joint, which are used as robot processing information.

[0073] In some optional implementations of certain embodiments, the aforementioned execution entity may generate robot processing information based on the aforementioned first control processing information, including the following steps: The first step, in response to the target control layer being an interactive action control layer, is to acquire the current robot processing information. When the target control layer is an interactive action control layer, the real-time input information can represent input information related to action execution. For example, the real-time input information could be "point out the direction of the cup." The current robot processing information can be the processing information output by each of the remaining control layers (excluding the target control layer) corresponding to the target robot at the current time. For example, if the target control layer is an interactive action control layer, the remaining control layers could include: a posture control layer and a micro-expression control layer. The summary information of the processing information corresponding to each control layer is the current robot processing information. Alternatively, the current robot processing information can also be the operation information corresponding to the processing operation to be performed by the target robot at the current time.

[0074] The second step involves obtaining the third weight change information corresponding to the first control processing information and the fourth weight change information corresponding to the current robot processing information. The third weight change information represents a weight change that gradually increases during the execution time period. The fourth weight change information represents a weight change that gradually decreases during the execution time period.

[0075] The third weight change information can be the change in the importance of each joint parameter in the joint parameter information sequence corresponding to the first control processing information. Each joint parameter can be the adjustment parameter value used by the target robot to adjust the posture of each joint during the execution time period. The joint parameter information sequence represents the posture changes at each execution time within the execution time period. That is, the third weight change information is a sequence of third weight values. There is a one-to-one correspondence between the third weight values ​​in the third weight value sequence and the joint parameter information in the joint parameter information sequence corresponding to the first control processing information. The execution time period can be the feedback time period during which the target robot provides feedback or response to real-time input information. The fourth weight change information can be the change in the importance of each joint parameter in the joint parameter information sequence corresponding to the information currently processed by the robot. That is, the fourth weight change information is a sequence of fourth weight values. There is a one-to-one correspondence between the fourth weight values ​​in the fourth weight value sequence and the joint parameter information in the joint parameter information sequence corresponding to the information currently processed by the robot.

[0076] Here, the third weight change information is set to gradually increase the weight change information, and the fourth weight change information is set to gradually decrease the weight change information. This is so that when the target robot is responding to real-time input information during the execution period, the corresponding action interaction content of the target robot should have a large weight. Subsequently, when the target robot changes its posture, it will focus on the interaction posture change of the action interaction.

[0077] The third step involves fusing the first control processing information and the current robot processing information based on the third weight change information and the fourth weight change information to obtain the robot processing information.

[0078] As an example, for each execution time within the execution time period, based on the third weight value in the third weight change information and the fourth weight value in the fourth weight change information corresponding to the execution time, the joint parameter information corresponding to the first control processing information and each joint parameter information in the current robot processing information are weighted to obtain joint fusion parameter information. The resulting sequence of joint fusion parameter information is then determined as the robot processing information.

[0079] In some optional implementations of certain embodiments, the steps further include: The first step, in response to the lack of input information in the robot's final state, is to obtain future posture processing information for the posture control layer in a future time period. Here, the robot's final state refers to the state after the processing operation. The lack of input information could mean the lack of the latest semantic control information, i.e., the lack of the latest input control command. The future time period can be a time period after the final state. The future posture processing information can be posture change information in the future; that is, the future posture processing information can be the content of the posture processing determined in the future.

[0080] The second step involves obtaining the fifth weight change information corresponding to the aforementioned future attitude processing information and the sixth weight change information corresponding to the first control processing sub-information. The first control processing sub-information refers to the control processing information within the aforementioned future time period. The fifth weight change information represents a gradually increasing weight change within the aforementioned future time period, and the sixth weight change information represents a gradually decreasing weight change within the aforementioned future time period. The fifth and sixth weight change information can be weight sequences within the future time period, representing changes in the importance of the corresponding processing information.

[0081] The third step involves taking the robot's final state as the starting state, and then, based on the fifth and sixth weight change information, fusing the future posture processing information and the first control processing sub-information to obtain the first future processing information.

[0082] As an example, the joint parameter states of each joint corresponding to the robot's final state can be taken as the starting state. Based on the fifth weight change information and the sixth weight change information, the joint parameters of the future posture processing information and the first control processing sub-information can be weighted and fused at the corresponding time to obtain the first future processing information.

[0083] The fourth step is to execute the processing operation corresponding to the first future processing information mentioned above.

[0084] In some optional implementations of certain embodiments, the steps further include: The first step involves acquiring micro-expression control information at the target future time, followed by obtaining micro-expression processing information output by the micro-expression control layer, the robot's future state at the target future time, and future processing sub-information. Specifically, for the target control layer (the aforementioned micro-expression control layer), the corresponding micro-expression processing information consists of joint parameter information corresponding to the micro-expression. This micro-expression processing information can be joint parameter adjustment information for the micro-expression. The target future time can be the point in time within the target future time period where the micro-expression control information is acquired. The robot's future state can be the robot state of the target robot at the target future time. That is, the robot's future state can be the joint parameter state corresponding to each joint of the target robot at the target future time. The future processing sub-information can be the processing sub-information following the target future time in the robot processing information. Since the robot processing information is a temporal sequence of joint parameters corresponding to each joint, each joint has a corresponding joint parameter sequence. The joint parameters in the joint parameter sequence have corresponding execution times. Therefore, the future processing sub-information here can be the joint parameter sub-sequence following the joint parameters corresponding to the target future time.

[0085] The second step involves obtaining the seventh weight change information corresponding to the aforementioned future processing sub-information and the eighth weight change information corresponding to the aforementioned micro-expression processing information. The seventh weight change information is a weight change that gradually decreases during the execution period, while the eighth weight change information is a weight change that gradually increases and then gradually decreases during the execution period. The eighth weight change information is a weight change that experiences an instantaneous weight peak triggered by an event and decays exponentially. For example, the eighth weight change information may be a weight change that momentarily increases and then falls back according to an exponential or other decay function. The seventh weight change information can also exhibit a weight change trend opposite to the eighth weight change information. For example, if the eighth weight change information is a weight change that momentarily increases and then falls back according to an exponential or other decay function, the corresponding seventh weight change information could be a weight change that momentarily decreases and then increases.

[0086] The third step involves taking the robot's future state as the starting state, and then, based on the seventh and eighth weight change information, processing and fusing the future processing sub-information and the micro-expression processing information to obtain the second future processing information.

[0087] As an example, the aforementioned execution entity can take the joint parameter state corresponding to the robot's future state as the starting state, and perform joint parameter weighting processing on the future processing sub-information and micro-expression processing information based on the aforementioned seventh weight change information and the aforementioned eighth weight change information to obtain the joint parameter weighting result, which serves as the second future processing information.

[0088] The fourth step is to execute the processing operation corresponding to the second future processing information at the aforementioned target future time.

[0089] In some optional implementations of certain embodiments, the trigger threshold corresponding to the micro-expression control layer is determined based on the object sensitivity of the interactive object. The object sensitivity can be user sensitivity. Object sensitivity can characterize the sensitive value at which the micro-expression control layer triggers micro-expression processing. The trigger threshold can be a threshold for micro-expression generation. For example, for highly sensitive users (e.g., serious users), the corresponding trigger threshold can be increased. For low-sensitive users (e.g., lively users), the corresponding trigger threshold can be decreased.

[0090] Optionally, the aforementioned executing entity may obtain the seventh weight change information corresponding to the aforementioned future processing sub-information and the eighth weight change information corresponding to the aforementioned micro-expression processing information, including the following steps: The first step is to set weight decay information corresponding to the micro-expression processing information based on the aforementioned object sensitivity. This weight decay information can represent the weight decay of the micro-expression processing information during subsequent fusion processes. In other words, it can characterize the weight changes as the micro-expression appears and disappears completely.

[0091] As an example, you can query the weight decay information corresponding to an object's sensitivity from the sensitivity association table. The sensitivity association table can represent the relationship between object sensitivity and weight decay information.

[0092] The second step is to set the eighth weight change information corresponding to the aforementioned micro-expression processing information, based on the weight decay information described above. Details will not be elaborated further. The weight decay information can be directly determined as the fifth weight change information.

[0093] Optionally, based on the aforementioned object sensitivity, weight attenuation information corresponding to the aforementioned micro-expression processing information is set, including: The weight decay information corresponding to the above micro-expression processing information can be set based on the object's sensitivity and historical feedback. Historical feedback can refer to the feedback content given to micro-expression behavior in the past. For example, historical feedback could be exaggerated feedback content indicating discomfort in micro-expression expression.

[0094] In some optional implementations of certain embodiments, the execution entity may obtain the fifth weight change information corresponding to the future attitude processing information and the sixth weight change information corresponding to the first control processing sub-information, including the following steps: The first step is to determine the duration of weight changes based on the execution environment corresponding to the aforementioned future time periods. The duration of weight changes can be the total duration of weight changes over the future timeframe. A corresponding weight selection strategy can be determined based on the execution environment. Based on the weight selection strategy, the duration of the corresponding weight changes can be determined.

[0095] The second step is to set the fifth and sixth weight change information based on the duration of the aforementioned weight changes. This setting can be done within the duration limit of the weight changes and while ensuring that the fifth weight change information represents a gradually increasing weight change over the aforementioned future time period, and the sixth weight change information represents a gradually decreasing weight change over the aforementioned future time period.

[0096] In some optional implementations of certain embodiments, the execution entity may obtain the third weight change information corresponding to the first control processing information and the fourth weight change information corresponding to the current robot processing information, including the following steps: The first step is to obtain the corresponding weight selection strategy based on the execution environment within the aforementioned execution time period. The execution environment can refer to the environmental conditions within the execution time period. In practice, the execution environment may include scene information and the characteristics of the people within the scene. The weight selection strategy can be the strategy content for how to select the weight changes corresponding to the information processed by the robot. For example, for a scenario where the execution environment is for elderly users, the corresponding weight selection strategy tends to select weight changes for "accuracy of information transmission" and "professional actions." For a scenario where the execution environment is for users needing assistance, the corresponding weight selection strategy tends to select weight changes that increase the weight of approachable actions such as "empathic micro-expressions" and "leaning forward."

[0097] As an example, the aforementioned execution entity can query the weight selection strategy under the execution environment within the aforementioned execution time period by querying the mapping table between the environment and the selection strategy. The mapping table between the environment and the selection strategy can represent the precise mapping relationship between the weight selection strategies under different execution environments.

[0098] The second step is to set the third and fourth weight change information according to the weight selection strategy mentioned above.

[0099] As an example, the third and fourth weight change information mentioned above can be extracted from the weight selection strategy.

[0100] Step 204: Perform the processing operation corresponding to the robot processing information mentioned above.

[0101] In some embodiments, the aforementioned execution entity may perform processing operations corresponding to the aforementioned robot processing information.

[0102] As an example, the aforementioned execution entity can control the movement of each joint according to the joint parameters corresponding to the robot's processed information, so as to realize the execution of the processing operation corresponding to the robot's processed information.

[0103] See Figure 3 The diagram illustrates the weight changes of the posture control layer, interaction action control layer, and micro-expression control layer.

[0104] like Figure 3 As shown, in stage 1, the robot's processing information is primarily driven by the output of the posture control layer; the output information from the interaction action control layer and the micro-expression control layer is not yet available. Therefore, a high weight (e.g., 0.8) is assigned to the output content of the posture control layer, allowing the posture control layer to dominate the target robot's posture changes. In stage 2, the target robot receives action commands input from the interaction action control layer, which then outputs the corresponding information. The transition time between stage 1 and stage 2 is t1. Thus, within stage 2, after time t1, the weight of the posture control layer's output content decreases, while the weight of the interaction action control layer's output content increases and then decreases, reflecting the change in the interaction action control layer's output content. At time t2, the interaction action control layer's output content is complete, and stage 3 requires the fading of this output content. Therefore, in stage 3, the weight of the interaction action control layer's output content decreases, while the weight of the posture control layer's output content increases, switching the target robot from a state where the original action command has not yet occurred. At time point t3, stage 4 begins, where the posture control layer's output takes over completely to ensure the target robot returns to its original state. At time point t4, the micro-expression control layer's output is triggered, and the weight of its output changes from a surge to a decay, allowing for micro-expression adjustments while maintaining the posture control layer's output. At time point t5, the micro-expression control layer's output completely disappears.

[0105] The various embodiments of this disclosure have the following beneficial effects: Through the robot execution methods of some embodiments of this disclosure, continuous and natural changes in robot posture can be achieved when the robot posture meets the control requirements corresponding to the input information. Specifically, the reason why the changes in robot posture are not natural enough and do not meet the semantic execution requirements is that different types of motion compete for the same control resources, resulting in mutual overriding or cancellation. This may lead to the main execution content being overridden or cancelled, causing the content executed by the robot to mismatch with the execution requirements. Based on this, the robot execution method of some embodiments of this disclosure firstly, in response to the target robot obtaining real-time input information, determines the target control layer based on the real-time input information. Here, by setting a posture control layer, an interactive action control layer, and a micro-expression control layer, the control layer that needs to output the main content can be accurately located based on the different real-time input information, so as to achieve adaptive posture changes for different types of input information (i.e., different types of event processing requests). In addition, by determining the target control layer, the generation of the main execution semantic content can be realized, ensuring that the main execution semantic content is not overridden or cancelled during the subsequent execution of the target robot, thus ensuring the correctness of the target robot's corresponding execution semantics. Then, based on the aforementioned real-time input information, the target control layer generates first control processing information for use in generating the semantic content to be executed by the robot. Next, robot processing information is generated based on the first control processing information. Here, under the constraints of the first control processing information, and while ensuring that the semantic content of the execution does not deviate from the execution content corresponding to the first control processing information, the robot processing information can be accurately generated. Finally, the processing operation corresponding to the robot processing information is executed. In summary, by determining the target control layer based on real-time input information, and under the constraints of the semantic content of the first control processing information output by the target control layer, the robot can avoid deviating from the execution requirements corresponding to the real-time input information during subsequent execution, thus achieving continuous and natural changes in robot posture.

[0106] Further reference Figure 4 The diagram illustrates a flow 400 of another embodiment of a robot execution method according to the present disclosure. This robot execution method includes the following steps: Step 401: In response to obtaining real-time input information, determine the target control layer based on the real-time input information.

[0107] Step 402: Based on the real-time input information mentioned above, the first control processing information is generated using the target control layer mentioned above.

[0108] Step 403: Generate robot processing information based on the first control processing information mentioned above.

[0109] Step 404: Perform the processing operation corresponding to the robot processing information mentioned above.

[0110] In some embodiments, the specific implementation of steps 401-404 and the resulting technical effects can be found in [reference needed]. Figure 2 Steps 201-204 in the corresponding embodiments will not be repeated here.

[0111] Step 405: In response to obtaining intermediate input information during the processing operation and determining that the target control layer is an interactive action control layer, obtain the input time corresponding to the intermediate input information.

[0112] In some embodiments, in response to obtaining intermediate input information during the execution of a processing operation and determining that the target control layer is an interactive action control layer, the execution subject (e.g., Figure 1 The target robot 101 shown can obtain the input time corresponding to the aforementioned intermediate input information. The aforementioned input time is the time within the execution time period. The processing operation can be the process by which the target robot processes information according to the robot's processing information. The input information can be event-related input information received when the robot performs related processing. For example, when the target robot is performing a "clapping" action, its arms are already raised to the middle position. If the user triggers a new touch event, the robot needs to switch to a "head tilt response" action. Here, the input information can be the triggered new touch event.

[0113] Step 406: Obtain robot processing sub-information after the above-mentioned input time and robot execution status at the above-mentioned input time.

[0114] In some embodiments, the executing entity can acquire robot processing sub-information after the input time and the robot execution state at the input time. The robot processing sub-information can be a portion of the robot processing information after the input time. The robot execution state at the input time can be the robot state exhibited by the target robot at the input time. In practice, the robot execution state here can be the state of each joint parameter at the input time.

[0115] Step 407: Based on the above intermediate input information, generate second control processing information using the above interactive action control layer.

[0116] In some embodiments, the execution entity may generate second control processing information based on the intermediate input information and the interactive action control layer. The second control processing information may be adjustment content for the interactive action adjustments made in response to the input information.

[0117] As an example, input information can be fed into the interactive action control layer to leverage the large model within the interactive action control layer to generate secondary control processing information.

[0118] Step 408: Obtain the first weight change information corresponding to the second control processing information and the second weight change information corresponding to the robot processing sub-information.

[0119] In some embodiments, the execution entity may acquire first weight change information corresponding to the second control processing information and second weight change information corresponding to the robot processing sub-information. The first weight change information may be weight change information that is gradually increased and then decreased after the input time. The second weight change information may be weight change information that is gradually decreased.

[0120] As an example, the first weight change information corresponding to the second control processing information and the second weight change information corresponding to the robot processing sub-information can be dynamically set according to the execution environment corresponding to the robot's execution state.

[0121] Here, by setting first weight change information and second weight change information, the motion balance transition from robot processing sub-information to second control processing information is achieved.

[0122] Step 409: Taking the robot's execution state as the starting state, based on the first weight change information and the second weight change information, the second control processing information and the robot processing sub-information are processed and fused to obtain robot adjustment processing information.

[0123] In some embodiments, the execution entity may take the robot's execution state as the starting state, and perform information fusion on the second control processing information and the robot processing sub-information based on the first weight change information and the second weight change information to obtain robot adjustment processing information. Further details will not be elaborated further.

[0124] Step 410: Perform the processing operation corresponding to the robot adjustment processing information mentioned above.

[0125] In some embodiments, the aforementioned execution entity may perform the processing operations corresponding to the robot adjustment processing information. Further details will not be elaborated upon.

[0126] from Figure 4 It can be seen from this that, with Figure 2 Compared to the description of some corresponding embodiments, Figure 4In some corresponding embodiments, the robot execution method process 400 obtains intermediate input information during execution, takes the robot state at the input time corresponding to the intermediate input information as the robot's initial state, and realizes the response feedback of the intermediate input information by fusing the processing information between the second control processing information and the robot processing information.

[0127] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a robot execution device, which are similar to... Figure 2 Corresponding to the method embodiments shown, the robot actuator can be specifically applied to various electronic devices.

[0128] like Figure 5 As shown, a robot execution device 500 includes: a determining unit 501, a first generating unit 502, a second generating unit 503, and an execution unit 504. The determining unit 501 is configured to determine a target control layer based on real-time input information received; the first generating unit 502 is configured to generate first control processing information using the target control layer based on the real-time input information; the second generating unit 503 is configured to generate robot processing information based on the first control processing information; and the execution unit 504 is configured to execute the processing operation corresponding to the robot processing information.

[0129] In some optional implementations of certain embodiments, the robot execution device 500 further includes: in response to obtaining intermediate input information during the execution of a processing operation and determining that the target control layer is an interactive action control layer, obtaining the input time corresponding to the intermediate input information; obtaining robot processing sub-information after the input time and the robot execution state at the input time; generating second control processing information based on the intermediate input information and using the interactive action control layer; obtaining first weight change information corresponding to the second control processing information and second weight change information corresponding to the robot processing sub-information; using the robot execution state as a starting state, performing processing information fusion on the second control processing information and the robot processing sub-information based on the first weight change information and the second weight change information to obtain robot adjustment processing information; and executing the processing operation corresponding to the robot adjustment processing information.

[0130] In some optional implementations of certain embodiments, the second generation unit 503 may be further configured to: in response to the target control layer being an interactive action control layer, acquire current robot processing information; acquire third weight change information corresponding to the first control processing information and fourth weight change information corresponding to the current robot processing information, wherein the third weight change information is weight change information that gradually increases during the execution time period, and the fourth weight change information is weight change information that gradually decreases during the execution time period; and perform processing information fusion on the first control processing information and the current robot processing information based on the third weight change information and the fourth weight change information to obtain robot processing information.

[0131] In some optional implementations of certain embodiments, the second generation unit 503 may be further configured to: in response to the lack of input information in the robot's final state, acquire future posture processing information for the posture control layer to perform posture processing in a future time period, wherein the robot's final state is the state after the processing operation is performed; acquire the fifth weight change information corresponding to the future posture processing information and the sixth weight change information corresponding to the first control processing sub-information, wherein the first control processing sub-information is the control processing information in the future time period, the fifth weight change information is the weight change information that gradually increases in the future time period, and the sixth weight change information is the weight change information that gradually decreases in the future time period; take the robot's final state as the starting state, and perform processing information fusion on the future posture processing information and the first control processing sub-information according to the fifth weight change information and the sixth weight change information to obtain first future processing information; and execute the processing operation corresponding to the first future processing information.

[0132] In some optional implementations of certain embodiments, the second generation unit 503 may be further configured to: in response to obtaining micro-expression control information at a target future time, obtain micro-expression processing information output by the micro-expression control layer, the robot's future state at the target future time, and future processing sub-information; obtain the seventh weight change information corresponding to the future processing sub-information and the eighth weight change information corresponding to the micro-expression processing information, wherein the seventh weight change information is a weight change information that gradually decreases during the execution time period, and the eighth weight change information is a weight change information that gradually increases and then gradually decreases during the execution time period; taking the robot's future state as the starting state, perform processing information fusion on the future processing sub-information and the micro-expression processing information according to the seventh weight change information and the eighth weight change information to obtain second future processing information; and execute the processing operation corresponding to the second future processing information at the target future time.

[0133] In some optional implementations of some embodiments, the first generation unit 502 may be further configured to: generate personalized processing feature information based on the object profile of the interactive object, wherein the personalized processing feature information includes at least one of processing duration, processing range, processing speed and processing style; and generate first control processing information based on the personalized processing feature information and the real-time input information, using the target control layer.

[0134] In some optional implementations of some embodiments, the target control layer is a posture control layer; and the first generation unit 502 can be further configured to generate facial expression processing information as first control processing information using the posture control layer based on the processing speed, the processing style and the real-time input information.

[0135] In some optional implementations of some embodiments, the trigger threshold corresponding to the micro-expression control layer is determined based on the object sensitivity of the interactive object; and in some optional implementations of some embodiments, the second generation unit 503 can be further configured to: set the weight attenuation information corresponding to the micro-expression processing information according to the object sensitivity; and set the eighth weight change information corresponding to the micro-expression processing information according to the weight attenuation information.

[0136] In some optional implementations of some embodiments, the first generation unit 502 may be further configured to: determine the duration of weight change based on the execution environment corresponding to the future time period; and set the fifth weight change information and the sixth weight change information based on the duration of weight change.

[0137] In some optional implementations of some embodiments, the first generation unit 502 may be further configured to: obtain the corresponding weight selection strategy based on the execution environment within the execution time period; and set the third weight change information and the fourth weight change information based on the weight selection strategy.

[0138] It is understandable that the units described in the robot actuator 500 are consistent with the reference... Figure 2 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the robot actuator 500 and the units contained therein, and will not be repeated here.

[0139] The following is for reference. Figure 6 It shows a schematic diagram of the structure of an electronic device 600 suitable for implementing some embodiments of the present disclosure. Figure 6The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0140] like Figure 6 As shown, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. The random access memory 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, the read-only memory 602, and the random access memory 603 are interconnected via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0141] Typically, the following devices can be connected to the input / output interface 605: input devices 606 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 608 including, for example, magnetic tape, hard disk, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 6 Each box shown can represent a device or multiple devices as needed.

[0142] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a read-only memory 602. When the computer program is executed by the processing device 601, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0143] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0144] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0145] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: in response to acquiring real-time input information, determine a target control layer based on the real-time input information; generate first control processing information using the target control layer based on the real-time input information; generate robot processing information based on the first control processing information; and execute the processing operation corresponding to the robot processing information.

[0146] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a determining unit, a first generating unit, a second generating unit, and an execution unit. The names of these units do not necessarily limit the specific unit; for example, the execution unit may also be described as "a unit that performs the processing operations corresponding to the aforementioned robot processing information."

[0149] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0150] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the robot execution methods described above.

[0151] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A robot execution method, comprising: In response to receiving real-time input information, the target control layer is determined based on the real-time input information; Based on the real-time input information, the first control processing information is generated using the target control layer; Based on the first control processing information, robot processing information is generated; Perform the processing operation corresponding to the robot's processing information.

2. The method according to claim 1, wherein, The method further includes: In response to obtaining intermediate input information during the execution of the processing operation and determining that the target control layer is an interactive action control layer, the input time corresponding to the intermediate input information is obtained; Obtain robot processing sub-information after the input time and robot execution status at the input time; Based on the intermediate input information, the second control processing information is generated using the interactive action control layer; Obtain the first weight change information corresponding to the second control processing information and the second weight change information corresponding to the robot processing sub-information; Starting with the robot's execution state, the second control processing information and the robot processing sub-information are fused according to the first weight change information and the second weight change information to obtain robot adjustment processing information; Perform the processing operation corresponding to the robot adjustment processing information.

3. The method according to claim 1, wherein, The step of generating robot processing information based on the first control processing information includes: In response to the target control layer being an interactive action control layer, the current robot processing information is obtained; Obtain the third weight change information corresponding to the first control processing information and the fourth weight change information corresponding to the current robot processing information, wherein the third weight change information is the weight change information that gradually increases during the execution time period, and the fourth weight change information is the weight change information that gradually decreases during the execution time period; Based on the third weight change information and the fourth weight change information, the first control processing information and the current robot processing information are fused to obtain robot processing information.

4. The method according to claim 3, wherein, The method further includes: In response to the lack of input information in the robot's final state, the future attitude processing information of the attitude control layer for attitude processing in the future time period is obtained, wherein the robot's final state is the state after the processing operation is performed; The fifth weight change information corresponding to the future attitude processing information and the sixth weight change information corresponding to the first control processing sub-information are obtained, wherein the first control processing sub-information is the control processing information under the future time period, the fifth weight change information is the weight change information that gradually increases within the future time period, and the sixth weight change information is the weight change information that gradually decreases within the future time period. Taking the robot's last state as the starting state, the future posture processing information and the first control processing sub-information are fused according to the fifth weight change information and the sixth weight change information to obtain the first future processing information; Execute the processing operation corresponding to the first future processing information.

5. The method according to claim 4, wherein, The method further includes: In response to obtaining micro-expression control information at the target future time, the micro-expression processing information output by the micro-expression control layer, the robot's future state at the target future time, and the future processing sub-information are obtained. Obtain the seventh weight change information corresponding to the future processing sub-information and the eighth weight change information corresponding to the micro-expression processing information, wherein the seventh weight change information is the weight change information that gradually decreases during the execution time period, and the eighth weight change information is the weight change information that gradually increases and then gradually decreases during the execution time period. Starting with the robot's future state, the future processing sub-information and the micro-expression processing information are fused according to the seventh weight change information and the eighth weight change information to obtain the second future processing information; The processing operation corresponding to the second future processing information is executed at the target future time.

6. The method according to claim 1, wherein, The target control layer is a control layer in the control layer set used for superimposed control that focuses on processing the real-time input information. The control layer set includes: a posture control layer based on overall emotion, an interactive action control layer, and a micro-expression control layer. Each control layer in the control layer set is a control layer that adjusts the robot's state at different time scales. The posture control layer is a minute-level posture control layer, the interactive action control layer is a second-level action interaction processing layer, and the micro-expression control layer is a millisecond-level micro-expression processing layer.

7. The method according to claim 1, wherein, The step of generating first control processing information based on the real-time input information using the target control layer includes: Based on the object profile of the interactive object, personalized processing feature information is generated, wherein the personalized processing feature information includes at least one of processing duration, processing range, processing speed and processing style; Based on the personalized processing feature information and the real-time input information, the first control processing information is generated using the target control layer.

8. The method according to claim 7, wherein, The target control layer is an attitude control layer; as well as The step of generating first control processing information based on the personalized processing feature information and the real-time input information using the target control layer includes: Based on the processing speed, the processing style, and the real-time input information, the posture control layer generates facial expression processing information as the first control processing information.

9. The method according to claim 5, wherein, The trigger threshold corresponding to the micro-expression control layer is determined based on the object sensitivity of the interactive object; as well as The step of obtaining the seventh weight change information corresponding to the future processing sub-information and the eighth weight change information corresponding to the micro-expression processing information includes: Based on the object sensitivity, set the weight decay information corresponding to the micro-expression processing information; Based on the weight decay information, the eighth weight change information corresponding to the micro-expression processing information is set.

10. The method according to claim 4, wherein, The step of obtaining the fifth weight change information corresponding to the future attitude processing information and the sixth weight change information corresponding to the first control processing sub-information includes: The duration of the weight change is determined based on the execution environment corresponding to the future time period. Based on the duration of the weight change, fifth and sixth weight change information are set.

11. The method according to claim 3, wherein, The step of obtaining the third weight change information corresponding to the first control processing information and the fourth weight change information corresponding to the current robot processing information includes: Based on the execution environment within the specified execution time period, obtain the corresponding weight selection strategy; According to the weight selection strategy, the third weight change information and the fourth weight change information are set.

12. A robot execution device, comprising: The determining unit is configured to determine the target control layer based on the real-time input information received. The first generation unit is configured to generate first control processing information based on the real-time input information and using the target control layer. The second generation unit is configured to generate robot processing information based on the first control processing information; The execution unit is configured to perform the processing operations corresponding to the robot processing information.

13. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-11.

14. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.

Images