Humanoid robot behavior data processing method and related device

Abstract

The embodiment of the invention provides a humanoid robot behavior data processing method and related device.The method comprises the steps that in response to a first selection operation of a user for a first control of a first action label in an action interaction interface, a first resource calling instruction of the first action label is generated, the first resource calling instruction is received and transmitted to the cloud server, the cloud server is used for querying a first action resource according to the first resource calling instruction and transmitting the first action resource to the humanoid robot, and the first action resource is used for representing action data needed by the humanoid robot for completing a first action; acquiring an action execution feedback message from the cloud server; and displaying the action execution feedback message. The action library of the humanoid robot in the cloud server is started and called through the terminal equipment, multiple functional components of the humanoid robot are in one-key linkage, complex actions are completed, and the intelligence of behavior data processing of the humanoid robot is improved.

Description

Technical Field

[0001] This application belongs to the field of humanoid robot data processing, specifically relating to a method and related apparatus for processing humanoid robot behavior data. Background Technology

[0002] Currently, the field of humanoid robots is developing rapidly. Operators can hold handheld devices and easily control humanoid robots to perform basic actions, such as fist bumps, clapping, extending a hand, and shaking hands, through the control interface on the handheld screen and the physical controls of the handheld device itself. However, the types of behaviors that humanoid robots can currently perform are limited and cannot meet the complex interactive needs of humanoid robots in real-world scenarios. Summary of the Invention

[0003] This application provides a method and related apparatus for processing humanoid robot behavior data. By starting and calling the humanoid robot's motion library in the cloud server through a terminal device, multiple functional components of the humanoid robot can be linked with one click to complete complex actions, which is beneficial to improving the intelligence of humanoid robot behavior data processing.

[0004] In a first aspect, embodiments of this application provide a method for processing humanoid robot behavior data, applied to a terminal device in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. The method includes: In response to the user's first selection operation of the first control of the first action label in the action interaction interface, a first resource call instruction for the first action label is generated and transmitted to the cloud server. The cloud server is used to query the first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action. Obtain an action execution feedback message from the humanoid robot, the action execution feedback message including current action execution state information, the current action execution state information being used to characterize the current state of the humanoid robot performing the first action; Display the feedback message for the execution of the action.

[0005] In one possible example, generating a first resource invocation instruction for the first action label in response to a user's first selection operation on a first control of a first action label in the action interaction interface includes: Based on the first action tag, at least one functional component required for the humanoid robot to complete the first action is determined. The functional component includes at least one of the following: robot dual arm functional component, robot dexterous hand functional component, robot head movement functional component, voice broadcasting functional component, and expression switching functional component. Determine the resource list corresponding to at least one functional component; Based on the at least one functional component and the resource list, a first resource invocation instruction for the first action tag is generated.

[0006] In one possible example, determining the resource list corresponding to at least one functional component includes: determining whether the first action is a combined action; If it is determined that the first action is a combined action, then it is determined whether the cloud server stores the combined action sequence file corresponding to the first action based on the action sequence file set of the PTZ server. If it is determined that the cloud server does not store the combined action sequence file corresponding to the first action, then the first action is split into multiple sub-actions, each sub-action corresponding to a functional component. The first action sequence file corresponding to each sub-action is determined, resulting in multiple first action sequence files, which are then added to the resource list. If it is determined that the cloud server stores a combined action sequence file corresponding to the first action, then the combined action sequence file is added to the resource list.

[0007] In one possible example, after transmitting the first resource invocation instruction to the cloud server, the process includes: An action execution control interface for generating the first action tag is provided. The action execution control interface includes an action control sub-area, which includes a second control and a third control. The second control is used to control the humanoid robot to pause the execution of the first action and to control the humanoid robot to start the execution of the first action. The interface for controlling the execution of the action is displayed.

[0008] In one possible example, the action execution control interface includes an execution status sub-area and an error message sub-area, and the display of the action execution feedback message includes: Based on the action execution feedback message, determine the current action execution status information and error message; The current action execution status information is displayed in the execution status sub-area; The error message is displayed in the error message sub-area. In one possible example, the action execution control interface includes a running status sub-area, and the method further includes: Obtain operational status information from the humanoid robot, the operational status information being used to characterize the operational status of the humanoid robot's program; The running status information is displayed in the running status sub-area.

[0009] In one possible example, after responding to a user's first selection of a first control on a first action label in the action interaction interface, the method further includes: Obtain action resource change messages from cloud servers, wherein the cloud servers are used to generate action resource change messages after detecting updated action data, and to send the action resource change messages to the cloud servers; If it is determined from the action resource change message that the cloud server has updated the action data corresponding to the first action tag, then a second resource call instruction for the first action tag is generated based on the action resource change message and the first action tag. The second resource call instruction is transmitted to the cloud server.

[0010] Secondly, embodiments of this application provide a humanoid robot behavior data processing device, applied as a terminal device in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. The humanoid robot behavior data processing device includes an acquisition unit, a determination unit, and a construction unit, wherein... The processing unit is configured to respond to a user’s first selection operation on a first control of a first action label in the action interaction interface, generate a first resource call instruction for the first action label, and transmit the first resource call instruction to the cloud server. The cloud server is configured to query a first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action. The acquisition unit is used to acquire an action execution feedback message from the humanoid robot. The action execution feedback message includes current action execution state information, which is used to characterize the current state of the humanoid robot performing the first action. The display unit is used to display the action execution feedback message.

[0011] A third aspect of this application provides an electronic device, including: a processor and a memory; and one or more programs stored in the memory and configured to be executed by the processor, the programs including instructions for some or all of the steps as described in the first aspect.

[0012] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform some or all of the steps described in the first aspect of this application.

[0013] A fifth aspect of this application provides a computer program product, comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of this application. This computer program product may be a software installation package.

[0014] As can be seen, in this embodiment, the terminal device first responds to the user's first selection operation of the first control of the first action label in the action interaction interface, generates a first resource call instruction for the first action label, and transmits the first resource call instruction to the cloud server. The cloud server queries the first action resource according to the first resource call instruction and transmits the first action resource to the humanoid robot. The first action resource is used to represent the action data required for the humanoid robot to complete the first action. Then, it obtains the action execution feedback message from the cloud server. The action execution feedback message includes the current action execution status information, which is used to represent the current state of the humanoid robot performing the first action. Finally, the action execution feedback message is displayed. By starting and calling the humanoid robot's action library in the cloud server through the terminal device, multiple functional components of the humanoid robot can be linked with one click to complete complex actions, which is beneficial to improving the intelligence of humanoid robot behavior data processing. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the architecture of a humanoid robot behavior data processing system provided in an embodiment of this application; Figure 2This is a flowchart illustrating a method for processing humanoid robot behavior data provided in an embodiment of this application; Figure 3 This is a schematic diagram of an action interaction interface provided in an embodiment of this application; Figure 4 This is a schematic diagram of a process for generating resource dispatch orders provided in an embodiment of this application; Figure 5 This is a flowchart illustrating a process for determining a resource list, as provided in an embodiment of this application. Figure 6 This is a flowchart illustrating an action execution control interface provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; Figure 8 This is a block diagram of the functional units of a humanoid robot behavior data processing device provided in an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0018] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] In the embodiments of this application, "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone; A and B exist simultaneously; B exists alone. Among them, A and B can be singular or plural.

[0021] In this embodiment, the symbol " / " can indicate that the preceding and following objects are in an "or" relationship. Alternatively, the symbol " / " can also represent a division sign, i.e., performing a division operation. For example, A / B can mean A divided by B.

[0022] In the embodiments of this application, "at least one item" or its similar expression refers to any combination of these items, including any combination of a single item or a plurality of items. "One or more" means one or more, while "multiple" means two or more. For example, "at least one item" of a, b, or c can represent the following seven cases: a, b, c; a and b; a and c; b and c; a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0023] In the embodiments of this application, "equal to" can be used with "greater than" and is applicable to technical solutions used when "greater than" is used; it can also be used with "less than" and is applicable to technical solutions used when "less than" is used. When "equal to" is used with "greater than", it is not used with "less than"; when "equal to" is used with "less than", it is not used with "greater than".

[0024] To better understand the solutions of the embodiments of this application, the electronic devices, related concepts and background that may be involved in the embodiments of this application will be introduced below.

[0025] The electronic device described in this application embodiment is a device with wireless communication capabilities, and may be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal device, vehicle-mounted terminal device, industrial control terminal device, UE unit, UE station, mobile station, remote station, remote terminal device, mobile device, UE terminal device, wireless communication device, UE agent, or UE device, etc. The terminal device can be fixed or mobile. It should be noted that the terminal device can support at least one wireless communication technology, such as LTE, New Radio (NR), Wideband Code Division Multiple Access (WCDMA), etc. For example, terminal devices can be mobile phones, tablets, desktop computers, laptops, all-in-one computers, in-vehicle terminals, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, electronic devices or other processing devices connected to a wireless modem, wearable devices, terminal devices in future mobile communication networks, or terminal devices in future evolved public land mobile networks (PLMNs), etc.

[0026] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a humanoid robot behavior data processing system provided in an embodiment of this application. Figure 1As shown, the humanoid robot behavior data processing system 1 includes a humanoid robot 10, a cloud server 20, and a terminal device 30. The terminal device 30 is connected to the humanoid robot 10 and the cloud server 20, respectively. The cloud server 20 is connected to the humanoid robot 10 and is used to store the action data of the humanoid robot 10.

[0027] Among them, the humanoid robot 10 is an intelligent robot with a human appearance and limb structure (such as head, torso, dexterous hands, and limbs). It simulates human actions and interactions through multimodal sensors, drive systems, AI algorithms, and large language models to complete anthropomorphic tasks.

[0028] Among them, the cloud server 20 can store the humanoid robot's motion library. The motion library stores and manages a collection of motion sequence files of various preset actions and movement patterns, which is the foundation for realizing anthropomorphic behavior. It standardizes limb movements through algorithms, which can cover basic actions (walking, grasping), compound actions (opening doors, going up and down stairs), and interactive actions (waving, facial expression simulation), etc.

[0029] Among them, the terminal device 30 can be a handheld console, through which users can control the humanoid robot to perform certain actions.

[0030] Among them, the humanoid robot 10 is equipped with a heartbeat signal and error code LOG mechanism, which can continuously detect the healthy operation status of the program.

[0031] In one possible example, terminal device 30 first responds to the user's first selection operation of the first control of the first action label in the action interaction interface, generates a first resource call instruction for the first action label, and transmits the first resource call instruction to cloud server 20. Cloud server 20 uses the first resource call instruction to query the first action resource and transmits the first action resource to humanoid robot 10. The first action resource is used to represent the action data required for humanoid robot 10 to complete the first action. Next, terminal device 30 obtains action execution feedback messages from cloud server 20. The action execution feedback messages include current action execution status information, which is used to represent the current state of humanoid robot 10 executing the first action. Finally, terminal device 30 displays the action execution feedback messages. By starting and calling the action library of humanoid robot 10 in cloud server 20 through terminal device 30, multiple functional components of humanoid robot 10 can be linked with one click to complete complex actions, which is beneficial to improving the intelligence of humanoid robot 10's behavior data processing.

[0032] Please see Figure 2 , Figure 2This is a flowchart illustrating a humanoid robot behavior data processing method provided in an embodiment of this application. It is applied to a terminal device in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. The method includes: Step S201: In response to the user's first selection operation of the first control of the first action label in the action interaction interface, a first resource call instruction for the first action label is generated and transmitted to the cloud server. The cloud server is used to query the first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action.

[0033] The action interaction interface is used by users to interact with the humanoid robot through actions. The action interaction interface includes multiple controls with multiple action labels.

[0034] Among them, the action tag is the tag for the humanoid robot's human-like interactive actions, which is not limited here. The interactive action can be linked to five functional components: robot's two arm functional components, robot's dexterous hand functional components, robot's head movement functional components, voice broadcasting functional components, and expression switching functional components. For example, it can be a greeting action, an item delivery action, a closing interaction action, etc., which can be set according to actual needs.

[0035] Each action label can correspond to at least one action sequence file. For example, a control for the "greeting" action label: when a user touches this control on the action interaction interface, the analysis shows that the greeting action requires the humanoid robot to raise its head from its initial head-down position and turn horizontally in the direction of the user, and its facial expression to switch from "no expression" to "smiling expression," as well as to simultaneously broadcast the greeting "Hello, I am xxx, nice to meet you." The involved linkage function components include three function components: robot head movement function component, voice broadcast function component, and expression switching function component. It is necessary to call up the action sequence files corresponding to the three types of functions: head turning, expression switching, and voice transmission.

[0036] The first action resource includes at least one action sequence file corresponding to the action tag.

[0037] The cloud server stores the motion sequence files of five functional components: robot dual-arm functional components, robot dexterous hand functional components, robot head movement functional components, voice broadcasting functional components, and expression switching functional components.

[0038] Please see Figure 3 , Figure 3 This is a schematic diagram of an action interaction interface provided in an embodiment of this application. The action interaction interface includes a robot display sub-area and an action control display sub-area. The robot display sub-area is used to display the virtual image and identifier corresponding to the current humanoid robot. The identifier of the current humanoid robot is "Robot 002". The action control display sub-area displays the "greeting action" control, the "item delivery action" control, the "closing interaction action" control, and the "other interaction action" control.

[0039] Step S202: Obtain an action execution feedback message from the humanoid robot. The action execution feedback message includes current action execution state information, which is used to characterize the current state of the humanoid robot performing the first action.

[0040] After receiving the first action resource, the humanoid robot executes the first action and reports the current execution status to the terminal device so that users can monitor the status of the humanoid robot's actions in real time.

[0041] Step S203: Display the action execution feedback message.

[0042] The terminal device displays action execution feedback messages for the user to view.

[0043] As can be seen, in this embodiment, the terminal device first responds to the user's first selection operation of the first control of the first action label in the action interaction interface, generates a first resource call instruction for the first action label, and transmits the first resource call instruction to the cloud server. The cloud server queries the first action resource according to the first resource call instruction and transmits the first action resource to the humanoid robot. The first action resource is used to represent the action data required for the humanoid robot to complete the first action. Then, it obtains the action execution feedback message from the cloud server. The action execution feedback message includes the current action execution status information, which is used to represent the current state of the humanoid robot performing the first action. Finally, the action execution feedback message is displayed. By starting and calling the humanoid robot's action library in the cloud server through the terminal device, multiple functional components of the humanoid robot can be linked with one click to complete complex actions, which is beneficial to improving the intelligence of humanoid robot behavior data processing.

[0044] Please see Figure 4 , Figure 4 This is a flowchart illustrating the generation of a resource call command according to an embodiment of this application. Regarding the generation of a first resource call command for the first action tag in response to a user's first selection operation of a first control on a first action tag in an action interaction interface, the above method may include the following steps: Step S401: Based on the first action tag, determine at least one functional component required for the humanoid robot to complete the first action. The functional component includes at least one of the following: robot dual arm functional component, robot dexterous hand functional component, robot head movement functional component, voice broadcasting functional component, and expression switching functional component. Step S402: Determine the resource list corresponding to at least one functional component; Step S403: Generate a first resource invocation instruction for the first action tag based on the at least one functional component and the resource list.

[0045] In this example, a humanoid robot needs to coordinate at least one functional component to perform an action. For instance, in the action of delivering an item, the robot's arms extend forward in a coordinated manner around the shoulder joint, presenting a water cup to the user. The upper arm remains stable, while the forearm slightly adjusts its angle to ensure the cup is level. The fingers (thumb, index, and middle fingers) holding the cup are slightly relaxed for easy access, while the ring and little fingers are naturally raised to simulate the detail of "gentle delivery." The head moves slightly forward in sync with the arm extension, focusing its gaze between the cup and the user's hand, creating a state of "attentive to the delivery process." The robot maintains a smiling expression, but the eye animation incorporates a "slight blink" to enhance naturalness. A voice prompt reminds the user: "Please handle carefully, the water temperature is just right." Therefore, the action of delivering an item involves the coordinated functional components of the robot's arms, the robot's dexterous hand, the robot's head movement, the voice prompt, and the expression switching.

[0046] The resource list includes the action sequence file corresponding to each functional component involved in the first action.

[0047] The first resource call instruction is used to call the action sequence file in the resource list and give it to the humanoid robot.

[0048] As can be seen, in this embodiment of the application, the functional components required for the first action can be analyzed, and the action sequence files to be called by each functional component can be determined to obtain a list of team members. Then, a first resource call instruction is generated. By calling the action sequence files stored on the cloud server, the humanoid robot can perform the first action, which is conducive to improving the intelligence of the humanoid robot behavior data processing system.

[0049] Please see Figure 5 , Figure 5 This is a flowchart illustrating a method for determining a resource list according to an embodiment of this application. In determining the resource list corresponding to at least one functional component, the method may include the following steps: Step S501: Determine whether the first action is a combined action; Step S502: If it is determined that the first action is a combined action, then determine whether the cloud server stores a combined action sequence file corresponding to the first action based on the action sequence file set of the PTZ server. Step S503: If it is determined that the cloud server does not store the combined action sequence file corresponding to the first action, then the first action is split into multiple sub-actions, and each sub-action corresponds to a functional component. Step S504: Determine the first action sequence file corresponding to each sub-action to obtain multiple first action sequence files, and add the multiple first action sequence files to the resource list; Step S505: If it is determined that the cloud server stores a combined action sequence file corresponding to the first action, then the combined action sequence file is added to the resource list.

[0050] The action sequence file set includes multiple combined action sequence files, and each combined action sequence file includes at least two action sequence files.

[0051] Among them, a combined action refers to an action involving at least two functional components. The action sequence file corresponding to a single functional component can be called directly. However, for combined actions, it is necessary to consider whether the cloud server stores the combined action sequence file corresponding to the combined action. If the cloud server stores the combined action sequence file corresponding to the combined action, the combined action sequence file can be called. If the cloud server stores the combined action sequence file corresponding to the combined action, the first action is a user-edited custom combined action. The PTZ server has not yet updated the combined action sequence file corresponding to the custom action, and it is necessary to manually obtain the action sequence file of each functional component.

[0052] For example, the closing interaction includes the following actions: Arm movement: After the user takes the water cup, the arms slowly return to their initial raised position; Dexterous hand: The fingers are released, returning to an "empty hand" state with palms facing upwards, then naturally clenched into fists; Head movement: The head is raised again and turned towards the user's face, returning to the "looking at the user" angle; Expression switching: The smiling expression deepens, switching to a "happy expression"; Voice broadcast: Finally, it says, "If you have any other needs, just call me anytime~ Have a good time!" The closing interaction is a combined action, which can be broken down into two-wall movement actions, dexterous hand actions, head movement actions, expression switching actions, and voice broadcast actions. The two-wall movement actions are implemented by the robot's two-arm functional components, the dexterous hand actions are implemented by the robot's dexterous hand functional components, the head movement actions are implemented by the robot's head movement functional components, the expression switching actions are implemented by the voice broadcast functional components, and the voice broadcast actions are implemented by the expression switching functional components.

[0053] As can be seen, in the embodiments of this application, analyzing whether the first action is a combined action, and when the first action is a combined action, splitting the first action and determining the action sequence files corresponding to the sub-actions respectively, is beneficial to improving the intelligence of the humanoid robot behavior data processing system.

[0054] In one possible example, after transmitting the first resource call instruction to the cloud server, the process includes: generating an action execution control interface for the first action tag, the action execution control interface including an action control sub-area, the action control sub-area including a second control and a third control, the second control being used to control the humanoid robot to pause the execution of the first action, the second control being used to control the humanoid robot to start the execution of the first action; and displaying the action execution control interface.

[0055] The action execution control interface may include multiple sub-areas, and the action control sub-area includes a second control and a third control. The second control and the third control allow the user to pause the humanoid robot from performing the current action or start the humanoid robot from performing the current action in a "soft emergency stop" manner on the action execution control interface of the terminal device.

[0056] As can be seen in this example, the terminal device can generate and display an action execution control interface including an action control sub-area, so that users can control the humanoid robot's actions in real time, which is beneficial to improving the intelligence of the humanoid robot behavior data processing system.

[0057] In one possible example, the action execution control interface includes an execution status sub-area and an error message sub-area. Displaying the action execution feedback message includes: determining the current action execution status information and error message information based on the action execution feedback message; displaying the current action execution status information in the execution status sub-area; and displaying the error message information in the error message sub-area.

[0058] Optionally, the action execution control interface includes a running status sub-area, and the method further includes: acquiring running status information from the humanoid robot, the running status information being used to characterize the running status of the humanoid robot program; and displaying the running status information in the running status sub-area.

[0059] When the humanoid robot performs an action, errors may occur. The action execution control interface includes an execution status sub-area, an error message sub-area, and a running status sub-area. The execution status sub-area displays the status of the humanoid robot's current action, for example: Action xx in progress: Moved 0.5 meters. Action xx executed successfully: Object grasped. Action xx executed unsuccessfully: Error code E022: Path blocked.

[0060] The humanoid robot is equipped with a heartbeat signal and an error code / log mechanism. The heartbeat signal works by periodically sending a "heartbeat signal" to the monitoring system or log. If the monitoring system does not receive a heartbeat signal within a set time, it determines that the program may have crashed or frozen. The error code / log mechanism assigns a displacement code to each possible error; for example, E001 indicates a sensor connection failure. When an error occurs, the humanoid robot system records the error code, timestamp, and context information.

[0061] For an example, please refer to Figure 6 , Figure 6 This is a flowchart illustrating an action execution control interface provided in an embodiment of this application. The action execution control interface includes an action control sub-area, an execution status sub-area, an error message sub-area, and a robot display sub-area. The execution status sub-area currently displays the information "Action xx is being executed: moved 0.5 meters". The error message sub-area currently displays the information "Error code E022: 8:05:36 PM, path blocked". The robot display sub-area displays a virtual image of the robot numbered 002. The action control sub-area displays pause and start controls.

[0062] In one possible example, after responding to the user's first selection operation on the first control of the first action label in the action interaction interface, the method further includes: obtaining an action resource change message from a cloud server, wherein the cloud server is used to generate the action resource change message after detecting updated action data, and to send the action resource change message to the cloud server; if it is determined from the action resource change message that the cloud server has updated the action data corresponding to the first action label, then a second resource call instruction for the first action label is generated based on the action resource change message and the first action label; and the second resource call instruction is transmitted to the cloud server.

[0063] Among them, the cloud server supports online modification and updating of motion execution files. When the user updates the motion data of the humanoid robot in the cloud server, the cloud server can send motion resource change messages to the terminal device.

[0064] Among them, the action resource change message includes the identifier of the action execution file that has been changed.

[0065] Among them, the terminal device analyzes the action resource change message. If it finds that the cloud server has updated the action data corresponding to the first action tag, it means that the action data required for the humanoid robot to perform the first action has changed. It is necessary to determine a new second resource call instruction to call the updated action data.

[0066] As can be seen in this example, the cloud service supports online updates of the humanoid robot's motion data, and the terminal device updates the calling instructions in real time, eliminating the need for a cumbersome restart / loading process, which helps improve the intelligence of the humanoid robot behavior data processing system.

[0067] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, applied as a terminal device in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. Figure 7 As shown, the electronic device includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are configured to be executed by the processor according to the following instructions: In response to the user's first selection operation of the first control of the first action label in the action interaction interface, a first resource call instruction for the first action label is generated and transmitted to the cloud server. The cloud server is used to query the first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action. Obtain an action execution feedback message from the humanoid robot, the action execution feedback message including current action execution state information, the current action execution state information being used to characterize the current state of the humanoid robot performing the first action; Display the feedback message for the execution of the action.

[0068] As can be seen, in this embodiment, the electronic device first responds to the user's first selection operation of the first control of the first action label in the action interaction interface, generates a first resource call instruction for the first action label, and transmits the first resource call instruction to the cloud server. The cloud server queries the first action resource according to the first resource call instruction and transmits the first action resource to the humanoid robot. The first action resource is used to represent the action data required for the humanoid robot to complete the first action. Then, it obtains the action execution feedback message from the cloud server. The action execution feedback message includes the current action execution status information, which is used to represent the current state of the humanoid robot performing the first action. Finally, the action execution feedback message is displayed. By starting and calling the humanoid robot's action library in the cloud server through the terminal device, multiple functional components of the humanoid robot can be linked with one click to complete complex actions, which is beneficial to improving the intelligence of humanoid robot behavior data processing.

[0069] In one possible example, regarding the generation of a first resource invocation instruction for the first action label in response to a user's first selection operation of a first control for a first action label in the action interaction interface, the above-described program includes instructions for performing the following steps: Based on the first action tag, at least one functional component required for the humanoid robot to complete the first action is determined. The functional component includes at least one of the following: robot dual arm functional component, robot dexterous hand functional component, robot head movement functional component, voice broadcasting functional component, and expression switching functional component. Determine the resource list corresponding to at least one functional component; Based on the at least one functional component and the resource list, a first resource invocation instruction for the first action tag is generated.

[0070] In one possible example, regarding the determination of the resource list corresponding to at least one functional component, the above procedure includes instructions for performing the following steps: Determine whether the first action is a combination action; If it is determined that the first action is a combined action, then it is determined whether the cloud server stores the combined action sequence file corresponding to the first action based on the action sequence file set of the PTZ server. If it is determined that the cloud server does not store the combined action sequence file corresponding to the first action, then the first action is split into multiple sub-actions, each sub-action corresponding to a functional component. The first action sequence file corresponding to each sub-action is determined, resulting in multiple first action sequence files, which are then added to the resource list. If it is determined that the cloud server stores a combined action sequence file corresponding to the first action, then the combined action sequence file is added to the resource list.

[0071] In one possible example, after transmitting the first resource request instruction to the cloud server, the above procedure includes instructions for performing the following steps: An action execution control interface for generating the first action tag is provided. The action execution control interface includes an action control sub-area, which includes a second control and a third control. The second control is used to control the humanoid robot to pause the execution of the first action and to control the humanoid robot to start the execution of the first action. The interface for controlling the execution of the action is displayed.

[0072] In one possible example, regarding the determination of multiple objective fitting functions between the load power and loss power of the power supply module based on the power supply data, the above procedure further includes instructions for performing the following steps: Based on the action execution feedback message, determine the current action execution status information and error message; The current action execution status information is displayed in the execution status sub-area; The error message is displayed in the error message sub-area.

[0073] In one possible example, the action execution control interface includes a running status sub-area, and the program further includes instructions for performing the following steps: Obtain operational status information from the humanoid robot, the operational status information being used to characterize the operational status of the humanoid robot's program; The running status information is displayed in the running status sub-area.

[0074] In one possible example, after the user's first selection operation on a first control of a first action label in the action interaction interface, the above program further includes instructions for performing the following steps: Obtain action resource change messages from cloud servers, wherein the cloud servers are used to generate action resource change messages after detecting updated action data, and to send the action resource change messages to the cloud servers; If it is determined from the action resource change message that the cloud server has updated the action data corresponding to the first action tag, then a second resource call instruction for the first action tag is generated based on the action resource change message and the first action tag. The second resource call instruction is transmitted to the cloud server.

[0075] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0076] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0077] When dividing each function into modules according to its corresponding function. Figure 8 A functional unit block diagram of a humanoid robot behavior data processing device is given, such as... Figure 8 As shown, a terminal device is used in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. The humanoid robot behavior data processing device includes a processing unit 801, an acquisition unit 802, and a display unit 803. The processing unit 801 is configured to respond to the user's first selection operation on the first control of the first action label in the action interaction interface, generate a first resource call instruction for the first action label, and transmit the first resource call instruction to the cloud server. The cloud server is configured to query the first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action. The acquisition unit 802 is used to acquire an action execution feedback message from the humanoid robot. The action execution feedback message includes current action execution state information, which is used to characterize the current state of the humanoid robot performing the first action. The display unit 803 is used to display the action execution feedback message.

[0078] As can be seen, in this embodiment, the humanoid robot behavior data processing device first responds to the user's first selection operation on the first control of the first action label in the action interaction interface, generates a first resource call instruction for the first action label, and transmits the first resource call instruction to the cloud server. The cloud server queries the first action resource according to the first resource call instruction and transmits the first action resource to the humanoid robot. The first action resource is used to represent the action data required for the humanoid robot to complete the first action. Then, it obtains the action execution feedback message from the cloud server. The action execution feedback message includes the current action execution status information, which is used to represent the current state of the humanoid robot performing the first action. Finally, it displays the action execution feedback message. By starting and calling the humanoid robot's action library in the cloud server through the terminal device, multiple functional components of the humanoid robot can be linked with one click to complete complex actions, which is beneficial to improving the intelligence of humanoid robot behavior data processing.

[0079] In one possible example, regarding the generation of a first resource invocation instruction for the first action label in response to a user's first selection operation of a first control for a first action label in the action interaction interface, the processing unit 801 is specifically configured to: Based on the first action tag, at least one functional component required for the humanoid robot to complete the first action is determined. The functional component includes at least one of the following: robot dual arm functional component, robot dexterous hand functional component, robot head movement functional component, voice broadcasting functional component, and expression switching functional component. Determine the resource list corresponding to at least one functional component; Based on the at least one functional component and the resource list, a first resource invocation instruction for the first action tag is generated.

[0080] In one possible example, regarding the determination of the resource list corresponding to at least one functional component, the processing unit 801 is specifically used for: Determine whether the first action is a combination action; If it is determined that the first action is a combined action, then it is determined whether the cloud server stores the combined action sequence file corresponding to the first action based on the action sequence file set of the PTZ server. If it is determined that the cloud server does not store the combined action sequence file corresponding to the first action, then the first action is split into multiple sub-actions, each sub-action corresponding to a functional component. The first action sequence file corresponding to each sub-action is determined, resulting in multiple first action sequence files, which are then added to the resource list. If it is determined that the cloud server stores a combined action sequence file corresponding to the first action, then the combined action sequence file is added to the resource list.

[0081] In one possible example, after transmitting the first resource request instruction to the cloud server, the display unit 803 is specifically used for: An action execution control interface for generating the first action tag is provided. The action execution control interface includes an action control sub-area, which includes a second control and a third control. The second control is used to control the humanoid robot to pause the execution of the first action and to control the humanoid robot to start the execution of the first action. The interface for controlling the execution of the action is displayed.

[0082] In one possible example, the action execution control interface includes an execution status sub-area and an error message sub-area. The display unit 803, which displays the action execution feedback message, is further specifically used for: Based on the action execution feedback message, determine the current action execution status information and error message; The current action execution status information is displayed in the execution status sub-area; The error message is displayed in the error message sub-area.

[0083] In one possible example, regarding the inclusion of a running status sub-area in the action execution control interface, the display unit 803 is further specifically used for: Obtain operational status information from the humanoid robot, the operational status information being used to characterize the operational status of the humanoid robot's program; The running status information is displayed in the running status sub-area.

[0084] In one possible example, after the user's first selection operation on the first control of the first action label in the action interaction interface, the processing unit 801 is further specifically configured to: Obtain action resource change messages from cloud servers, wherein the cloud servers are used to generate action resource change messages after detecting updated action data, and to send the action resource change messages to the cloud servers; If it is determined from the action resource change message that the cloud server has updated the action data corresponding to the first action tag, then a second resource call instruction for the first action tag is generated based on the action resource change message and the first action tag. The second resource call instruction is transmitted to the cloud server.

[0085] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0086] The electronic device provided in this embodiment is used to execute the power allocation method described above, and therefore can achieve the same effect as the implementation method described above.

[0087] When using integrated units, the electronic device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the electronic device; for example, it can support the electronic device in executing the steps performed by the aforementioned functional units. The storage module can support the electronic device in executing stored program code and data. The communication module can support communication between the electronic device and other devices.

[0088] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.

[0089] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0090] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer includes a control platform.

[0091] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0092] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0094] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0097] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0098] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for processing humanoid robot behavior data, characterized in that, A terminal device is used in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. The method includes: In response to the user's first selection operation of the first control of the first action label in the action interaction interface, a first resource call instruction for the first action label is generated and transmitted to the cloud server. The cloud server is used to query the first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action. Obtain an action execution feedback message from the humanoid robot, the action execution feedback message including current action execution state information, the current action execution state information being used to characterize the current state of the humanoid robot performing the first action; Display the feedback message for the execution of the action.

2. The method according to claim 1, characterized in that, The step of generating a first resource invocation instruction for the first action label in response to a user's first selection operation on a first control of a first action label in the action interaction interface includes: Based on the first action tag, at least one functional component required for the humanoid robot to complete the first action is determined. The functional component includes at least one of the following: robot dual arm functional component, robot dexterous hand functional component, robot head movement functional component, voice broadcasting functional component, and expression switching functional component. Determine the resource list corresponding to at least one functional component; Based on the at least one functional component and the resource list, a first resource invocation instruction for the first action tag is generated.

3. The method according to claim 2, characterized in that, Determining the resource list corresponding to at least one functional component includes: Determine whether the first action is a combination action; If it is determined that the first action is a combined action, then it is determined whether the cloud server stores the combined action sequence file corresponding to the first action based on the action sequence file set of the PTZ server. If it is determined that the cloud server does not store the combined action sequence file corresponding to the first action, then the first action is split into multiple sub-actions, each sub-action corresponding to a functional component. The first action sequence file corresponding to each sub-action is determined, resulting in multiple first action sequence files, which are then added to the resource list. If it is determined that the cloud server stores a combined action sequence file corresponding to the first action, then the combined action sequence file is added to the resource list.

4. The method according to claim 1, characterized in that, After transmitting the first resource request instruction to the cloud server, the process includes: An action execution control interface for generating the first action tag is provided. The action execution control interface includes an action control sub-area, which includes a second control and a third control. The second control is used to control the humanoid robot to pause the execution of the first action and to control the humanoid robot to start the execution of the first action. The interface for controlling the execution of the action is displayed.

5. The method according to claim 4, characterized in that, The action execution control interface includes an execution status sub-area and an error message sub-area. The display of action execution feedback messages includes: Based on the action execution feedback message, determine the current action execution status information and error message; The current action execution status information is displayed in the execution status sub-area; The error message is displayed in the error message sub-area.

6. The method according to claim 4, characterized in that, The action execution control interface includes a running status sub-area, and the method further includes: Obtain operational status information from the humanoid robot, the operational status information being used to characterize the operational status of the humanoid robot's program; The running status information is displayed in the running status sub-area.

7. The method according to claim 1, characterized in that, After responding to a user's first selection operation on a first control of a first action label in the action interaction interface, the method further includes: Obtain action resource change messages from cloud servers, wherein the cloud servers are used to generate action resource change messages after detecting updated action data, and to send the action resource change messages to the cloud servers; If it is determined from the action resource change message that the cloud server has updated the action data corresponding to the first action tag, then a second resource call instruction for the first action tag is generated based on the action resource change message and the first action tag. The second resource call instruction is transmitted to the cloud server.

8. A humanoid robot behavior data processing device, characterized in that, A terminal device is used in a humanoid robot behavior data processing system. The humanoid robot behavior data processing system includes a humanoid robot, a cloud server, and the terminal device. The terminal device is connected to both the humanoid robot and the cloud server. The cloud server is connected to the humanoid robot and is used to store the humanoid robot's motion data. The humanoid robot behavior data processing device includes a processing unit, an acquisition unit, and a display unit. The processing unit is configured to respond to a user’s first selection operation on a first control of a first action label in the action interaction interface, generate a first resource call instruction for the first action label, and transmit the first resource call instruction to the cloud server. The cloud server is configured to query a first action resource according to the first resource call instruction and transmit the first action resource to the humanoid robot. The first action resource is used to characterize the action data required for the humanoid robot to complete the first action. The acquisition unit is used to acquire an action execution feedback message from the humanoid robot. The action execution feedback message includes current action execution state information, which is used to characterize the current state of the humanoid robot performing the first action. The display unit is used to display the action execution feedback message.

9. An electronic device, characterized in that, The method includes a processor, a memory, a communication interface, and one or more programs, said one or more programs being stored in the memory and configured to be executed by the processor, said programs including instructions for performing the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange, wherein the computer program causes a computer to perform the method as described in any one of claims 1-7.

Images