Method, device and equipment for controlling body-equipped robot and medium

Abstract

This invention relates to the field of artificial intelligence technology and relates to embodied robots applicable to smart healthcare and fintech. It discloses a control method, device, equipment, and medium for an embodied robot, comprising: collecting simulation data and real-world data in multiple scenarios; establishing a mapping relationship between the simulation data and the real-world data in each scenario; and generating fused data with mapping relationship metadata; responding to the embodied robot's target execution task; selecting and combining standardized algorithm modules from an algorithm template library according to the type and / or task scenario of the target execution task; and training the combined standardized algorithm modules using the fused data; wherein the algorithm template library stores standardized algorithm modules for performing different tasks and categorized by functional type and / or task scenario; and deploying the trained standardized algorithm modules to the embodied robot to drive it to execute the target execution task.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and natural language processing technology, and in particular to a control method, device, equipment and medium for an embodied robot. Background Technology

[0002] As a product of the deep integration of artificial intelligence (AI) and robotics, embodied robots have experienced rapid development in recent years. Unlike traditional industrial robotic arms, embodied robots possess multi-dimensional capabilities such as environmental perception, instruction understanding, autonomous decision-making, and physical interaction. They can serve complex and dynamic scenarios and have become a research hotspot in recent years, with widespread applications across various industries. For example, in the field of smart healthcare, embodied robots have been gradually applied to core scenarios such as surgical assistance, rehabilitation training, ICU monitoring, intelligent triage, and drug delivery. For instance, surgical assistance robots can acquire high-definition abdominal images in real time and identify lesion areas (such as tumors and bleeding points) to assist doctors in performing minimally invasive surgeries more accurately. In the fintech field, embodied robots have been implemented in scenarios such as intelligent customer service at bank branches, guidance at financial exhibition halls, and self-service business processing. They can understand customer intent through voice recognition and can also confirm customer identity by combining facial recognition, guiding customers to complete business processes and improving branch operational efficiency and customer experience.

[0003] However, in existing control methods for embodied robots, the implementation of control logic typically involves the coordination of multiple algorithm modules, including perception, decision-making, and drive execution. Currently, different algorithm modules are usually designed and developed by different teams, resulting in loosely defined data formats and varying interaction methods between modules. Integration requires significant manpower for interface adaptation and data format conversion. Furthermore, when algorithm modules trained in simulation environments are migrated to real robots, they are prone to issues such as control command deviations due to environmental factors, leading to a decrease in control efficiency and accuracy. Summary of the Invention

[0004] This invention provides a control method, device, equipment, and medium for embodied robots, to solve the technical problems that the control logic implementation of existing embodied robots requires a lot of adaptation work for multiple algorithm modules, and that after being migrated to the real robot, the control efficiency and accuracy are easily reduced due to environmental factors.

[0005] Firstly, a control method for an embodied robot is provided, including: Collect simulation data and real data from multiple scenarios, establish mapping relationships between the simulation data and real data in each scenario, and generate fused data with mapping relationship metadata; In response to the target execution task of the embodied robot, standardized algorithm modules are selected from the algorithm template library and combined according to the type and / or task scenario of the target execution task, and the combined standardized algorithm modules are trained using the fused data; wherein, the algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario; The trained standardized algorithm module is deployed to the android to drive the android to perform the target task.

[0006] Secondly, a control device for an embodied robot is provided, comprising: The data acquisition and processing module is used to acquire simulation data and real data in multiple scenarios, establish the mapping relationship between the simulation data and the real data in each scenario, and generate fused data with mapping relationship metadata. A training module is selected to respond to the target execution task of the embodied robot. Based on the type of the target execution task and / or the task scenario, standardized algorithm modules are selected from the algorithm template library and combined. The combined standardized algorithm modules are trained using the fused data. The algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario. A deployment control module is used to deploy the trained standardized algorithm module to the embodied robot to drive the embodied robot to perform the target execution task.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the control method for the embodied robot described above.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the control method for the embodied robot described above.

[0009] The aforementioned control method, device, equipment, and medium for embodied robots can collect simulation data and real-world data from multiple scenarios, establish mapping relationships between the simulation data and real-world data in each scenario, and generate fused data with mapping relationship metadata. It can also receive the embodied robot's target execution task through a client, select and combine standardized algorithm modules from an algorithm template library based on the type and / or task scenario of the target execution task. The algorithm template library stores standardized algorithm modules for performing different tasks and categorized by functional type and / or task scenario. The combined standardized algorithm modules are trained using the fused data. Finally, the trained standardized algorithm modules are deployed to the embodied robot to drive it to execute the target execution task. In this invention, for robots used for surgical assistance in smart healthcare scenarios or intelligent customer service robots in financial services, data from the corresponding scenarios can be collected. Simulation data is mapped to real-world data to generate fused data with mapping metadata, eliminating domain differences between virtual and real environments. Simultaneously, based on the type and / or scenario of the embodied robot's target task, appropriate standardized algorithm modules can be selected and combined from an algorithm template library containing standardized algorithm modules categorized by function type and / or task scenario for different tasks. The generated fused data with mapping metadata is then used to train the combined standardized algorithm modules. This avoids extensive adaptation and integration work caused by inconsistent interfaces and data protocols, effectively shortening the integration process. Furthermore, training with fused data improves the module's robustness to environmental changes, preventing performance degradation after migration to the embodied robot. It also eliminates the need for re-collecting large amounts of new scenario data for repeated training, allowing direct drive control, simplifying the overall control process, and improving control efficiency and accuracy. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for a control method of an embodied robot according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the control method for the embodied robot in the first embodiment of the present invention; Figure 3 yes Figure 2 A schematic diagram of a specific implementation of step S110; Figure 4yes Figure 3 A flowchart illustrating a specific implementation of step S111; Figure 5 This is another flowchart illustrating the control method of the embodied robot in the first embodiment of the present invention; Figure 6 yes Figure 2 A schematic diagram of a specific implementation of step S120; Figure 7 This is a flowchart illustrating the control method for the embodied robot in the second embodiment of the present invention; Figure 8 yes Figure 7 A schematic diagram of a specific implementation of step S250; Figure 9 This is a schematic diagram of the control device for a unibody robot according to an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 11 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The control method for the android provided in this embodiment of the invention can be applied to, for example... Figure 1In the application environment, the client communicates with the server via a network. The control method of the embodied robot of this invention can be configured to be executed on the server side, based on data interaction between the client and the server. First, the server can collect simulation data and real data in multiple scenarios, establish a mapping relationship between the simulation data and the real data in each scenario, and generate fused data with mapping relationship metadata. The server can also receive the target execution task of the embodied robot through the client. The server selects and combines standardized algorithm modules from the algorithm template library according to the type and / or task scenario of the target execution task. The algorithm template library stores standardized algorithm modules for executing different tasks and classified by functional type and / or task scenario. Then, the combined standardized algorithm modules are trained using the fused data of the corresponding scenario. Finally, the trained standardized algorithm modules are deployed to the embodied robot to drive and control the embodied robot to execute the target execution task. In this invention, for robots used in core scenarios such as surgical assistance, rehabilitation training, and intelligent triage in the field of smart healthcare, or intelligent customer service robots in financial business, data can be collected from the corresponding scenario. The simulation data and real-world data are mapped together to generate fused data with mapping metadata, eliminating domain differences between the virtual and real environments. Simultaneously, based on the type and / or scenario of the embodied robot's target task, appropriate standardized algorithm modules are selected and combined from an algorithm template library containing standardized algorithm modules categorized by function type and / or task scenario for different tasks. The generated fused data with mapping metadata is then used to train the combined standardized algorithm modules. This avoids extensive adaptation and integration work caused by inconsistent interfaces and data protocols, effectively shortening the integration chain. Furthermore, training with fused data improves the module's robustness to environmental changes, preventing performance degradation after migration to the embodied robot. It also eliminates the need for re-collecting large amounts of new scenario data for repeated training, allowing direct drive control, simplifying the overall control process, and improving control efficiency and accuracy. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention is described in detail below through specific embodiments.

[0014] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the control method for the embodied robot provided in the first embodiment of the present invention includes the following steps: S110: Collect simulation data and real data from multiple scenarios, establish a mapping relationship between the simulation data and the real data in each scenario, and generate fused data with mapping relationship metadata.

[0015] In this step, simulation data and real-world data are acquired for different task scenarios (such as surgical assistance and rehabilitation training in the field of smart healthcare, bank customer service and self-service processing in the field of fintech, and collaborative grasping and quality inspection in the industrial field). The simulation data and real-world data correspond to each other in each scenario. Simulation data refers to the data generated by running the corresponding task execution process in a virtual environment, which can include precisely labeled images, point clouds, force signals, etc. Real-world data refers to the data collected in real time by sensors (such as cameras, LiDAR, and microphone arrays) deployed on the real robot, including images, voice, and sensor readings in the actual environment. Taking the intelligent customer service scenario in bank branches as an example, simulation data can include facial images under different lighting, angles, and occlusion conditions generated by constructing a virtual environment of the business hall using 3D scene software, as well as voice commands with background noise, and corresponding voice responses and / or guidance action labels generated from the voice commands. Real-world data can include customer images, interactive voice data, and action commands collected by the camera and microphone array of the branch robot itself during actual customer service.

[0016] In some embodiments of the present invention, such as Figure 3 As shown, the step S110, which establishes the mapping relationship between the simulation data and the real data in each scenario, specifically includes the following steps S111-S112: S111: Perform multi-level alignment between the simulation data and the real data to establish spatial, temporal, and feature correspondences between the simulation data and the real data.

[0017] In this embodiment, for each task scenario, the simulation data and the real data in that scenario are aligned sequentially by time axis, spatial geometry, and feature distribution, thereby achieving multi-level alignment.

[0018] Specifically, such as Figure 4 As shown, the multi-level alignment of the simulation data and the real data in step S111 specifically includes the following steps S1111-S1113: S1111: Align simulation data and real-world data along the timeline.

[0019] In this step, a high-precision time synchronization protocol can be used to align the timestamps of simulation data and real data, establish a timestamp correspondence, and ensure that data from different environments strictly correspond on the timeline.

[0020] S1112: Extract the spatial co-occurrence features of the time-axis aligned simulation data and real data, and based on the spatial co-occurrence features, use a feature matching algorithm to perform spatial geometric alignment of the simulation data and real data.

[0021] Specifically, in this step, spatial co-occurrence features (such as object position, motion trajectory, point cloud contour, etc.) are extracted from the time-axis aligned simulation data and real data. A feature matching algorithm is used to perform spatial coordinate registration on the spatial co-occurrence features of the simulation data and the spatial co-occurrence features of the real data to obtain the spatial transformation relationship between the simulation data and the real data. Based on the spatial transformation relationship, the simulation data and the real data are spatially geometrically aligned.

[0022] In this embodiment, the feature matching algorithm may include Iterative ClosestPoint (ICP), Scale-Invariant Feature Transform (SIFT), Oriented Fast and Rotated BRIEF (ORB), and / or deep learning-based feature matching methods such as SuperGlue and D2-Net. In specific applications, the algorithm can be selected based on the data modality; for example, ICP is preferred for point cloud feature data, while SIFT, ORB, or deep learning-based matching methods are preferred for image feature data.

[0023] S1113: Align the feature distribution of the simulation data and the real data after spatial geometric alignment.

[0024] In this step, a domain adaptation method can be used to align the feature distribution of the spatially geometrically aligned simulation data and real data based on an adversarial domain adaptation network, thereby reducing the domain offset between the simulation data and the real data.

[0025] In some embodiments, the adversarial domain adaptation network can employ an adversarial neural network architecture, including a feature extractor (composed of a set of stacked convolutional or fully connected layers, used to map simulated and real data to a common feature space and output feature vectors), a label predictor (composed of fully connected layers and a softmax output layer, used to predict class labels based on the feature vectors of the simulated data output by the feature extractor), and a domain discriminator (composed of a binary classification network, used to indicate whether the input features come from simulated or real data). A gradient reversal layer (GRL) is placed between the feature extractor and the domain discriminator. This gradient reversal layer is used to invert the gradient from the domain discriminator and pass it to the feature extractor during backpropagation, so that the parameter update direction of the feature extractor is opposite to that of the domain discriminator, allowing the feature extractor to extract domain-invariant features that the domain discriminator cannot distinguish. The training objective of this network is to ensure that the features extracted by the feature extractor can both enable the label predictor to classify accurately and prevent the domain discriminator from distinguishing the source of the features, thereby achieving alignment between the simulated and real domains in the feature space, and completing the distribution alignment of simulated and real data in the feature space.

[0026] S112: Construct a mapping table indexed by task scenarios and task functions. The mapping table is used to associate simulation data, real data and their corresponding relationships.

[0027] In this embodiment, the correspondence between simulation data and real data includes the spatial transformation relationship, timestamp correspondence, and feature distribution alignment parameters between simulation data and real data. The feature distribution alignment parameters can be identifiers (such as model file names or version numbers) of pre-trained feature extractor models in the adversarial domain adaptation network, so that the model can be loaded for feature alignment after a query.

[0028] In this invention, the mapping table is a multi-level index structure with task scenarios (such as laparoscopic-assisted surgery) and task functions (such as lesion identification and instrument tracking) as index fields. In practical use, it supports both precise queries by specifying both task scenarios and task functions simultaneously, as well as queries by specifying only the task scenario or only the task function. This allows for quick retrieval of simulation data, real-world data, and the correspondence between them required for a specific scenario and / or task function. For example, when a user needs to obtain data corresponding to all task functions under the "laparoscopic surgery assistance" scenario, they can search using only the task scenario as the condition; when they need to obtain data for all "lesion identification" functions across scenarios, they can search using only the task function as the condition. The mapping table immediately locates the simulation data, real-world data, and the correspondence between them for that task function, thereby enabling on-demand assembly and efficient reuse of training data.

[0029] Specifically, in this step, generating fused data with mapping relationship metadata may include: querying the corresponding simulation data and real data and their corresponding relationships through the mapping table according to the task scenario and / or task function of the target task; performing temporal and spatial geometric alignment on the simulation data and real data sequentially based on spatial and temporal correspondences; then loading a pre-trained feature extractor model based on feature distribution alignment parameters to perform feature distribution alignment, further eliminating domain differences; finally, mixing the aligned simulation data and real data at a preset ratio (e.g., 7:3), and retaining the mapping relationship metadata between simulation data and real data, generating a fused dataset with mapping relationship metadata, thereby expanding the training set and maintaining training data consistency. In this embodiment, the fused data includes aligned simulation data and real data and their mapping relationship metadata; wherein, the mapping relationship metadata includes the spatial transformation relationship between simulation data and real data, the corresponding timestamp correspondence, and the feature distribution alignment parameters.

[0030] Based on the above steps, simulation data and real-world data from multiple scenarios are collected and a mapping relationship is established to generate fused data with mapping metadata. This eliminates the domain differences between the virtual and real environments, ensuring that the control logic implementation module trained based on the fused data maintains stable performance during cross-domain migration. It avoids repeated collection and retraining caused by the separation of virtual and real data, thereby improving the robustness of the control logic implementation module to environmental changes from the source.

[0031] S120: In response to the target execution task of the embodied robot, according to the type and / or task scenario of the target execution task, a standardized algorithm module is selected from the algorithm template library and combined, and the combined standardized algorithm module is trained using the fused data.

[0032] In this step, based on the task function type and / or task scenario of the target task, corresponding standardized algorithm modules are selected from the algorithm template library and combined. The selected and combined standardized algorithm modules are then trained using fused data corresponding to the scenario of the embodied robot's target task. The algorithm template library stores standardized algorithm modules for performing different tasks and categorized by function type and / or task scenario. Function types may include perception-based (e.g., image classification, object detection tasks), decision-making, execution, and interaction types; task scenario types may include industrial, financial, and medical scenarios.

[0033] Understandably, in some embodiments, such as Figure 5 As shown, the control method for the embodied robot may further include the following steps S100-S102: S100: Obtain the task execution flow of historical task scenarios, and break down the task execution flow of each historical task scenario to obtain multiple sub-tasks.

[0034] In this step, for multiple historical task scenarios that have already been responded to and executed, the task execution flow of each historical task scenario is obtained, and the task execution flow of each historical task scenario is broken down into multiple independent sub-tasks that can achieve different functions. For example, when a surgical robot in the field of smart healthcare performs a laparoscopic surgery assistance task, the laparoscopic surgery assistance task execution flow can be decomposed into multiple sub-tasks such as image acquisition, lesion detection, surgical instrument tracking, bleeding alarm, motion decision-making, and robotic arm control.

[0035] S101: Cluster the subtasks and merge subtasks with the same or similar functions into the same category.

[0036] In this step, subtasks under different historical task scenarios are clustered, and subtasks with the same or similar functions are merged into the same category.

[0037] S102: Encapsulate the algorithm flow corresponding to each type of subtask into a standardized algorithm module and store it in the algorithm template library.

[0038] In this step, each type of subtask is encapsulated into a standardized algorithm module. This module presents a unified standardized interface with a unified input / output data interface format and communication protocol, and is associated with the corresponding task scenario and / or task function. These modules are stored in the algorithm template library according to task scenario type and / or function type. Taking medical scenarios as an example, historical task scenarios such as laparoscopic surgery assistance or nodule medical diagnosis all contain subtasks for lesion detection, such as lesion identification. These can be clustered into the same category, and their corresponding algorithm flows can be encapsulated into a standardized algorithm module.

[0039] Furthermore, in some embodiments, before encapsulating the algorithm flow corresponding to each type of subtask into a standardized algorithm module, the method may further include: performing commonality analysis on all subtask flows of each type of subtask after clustering and merging to obtain common data flow patterns and common functional objectives of the internal execution flow of each type of subtask; and obtaining the algorithm flow corresponding to each type of subtask based on the common data flow patterns and common functional objectives. The data flow patterns may include the input and output data types of the subtasks and the internal process execution order, while the functional objectives may include tasks such as identity verification and risk identification. For example, in the context of intelligent customer service in the banking sector within the fintech field, the subtask of customer identity verification can include various specific implementations such as facial verification and voiceprint comparison. These can be encapsulated within the same standardized algorithm module. Before encapsulation, a commonality analysis is first performed on facial verification and voiceprint comparison to obtain their common data flow pattern: inputting multimodal data (facial images, voice, and ID photos) collected during customer interaction, performing feature extraction and comparison, and outputting verification results and confidence levels. The common functional objective is to confirm the consistency of customer identity. Based on this, a general corresponding algorithm flow can be developed, which includes input data adaptation, multimodal feature extraction, comparison decision, and verification output.

[0040] like Figure 6 As shown, step S120, which involves selecting and combining standardized algorithm modules from the algorithm template library based on the type and / or scenario of the target task, specifically includes the following steps S121-S122: S121: Based on the task function type and / or task scenario of the target execution task, determine the sub-tasks required to complete the target execution task, and select the standardized algorithm modules corresponding to each sub-task from the algorithm template library.

[0041] In this step, the target execution task is analyzed, and the target execution task can be divided into multiple sub-tasks based on the scenario type and / or the task function implemented. Standardized algorithm modules corresponding to each sub-task are selected from the algorithm template library.

[0042] S122: Determine the data dependencies and scheduling priorities among the selected standardized algorithm modules to form a combined standardized algorithm module.

[0043] In this invention, the data dependency relationship between standardized algorithm modules is the logical constraint and control flow generated by data transmission between the standardized algorithm modules, which determines the execution order of each standardized algorithm module.

[0044] Furthermore, in some embodiments of the present invention, when training the combined standardized algorithm modules using fused data, computing resources can be dynamically allocated according to scheduling priority, and high-performance computing resources (such as the core computing power of the central processing unit and GPU resources) and low-latency communication channels (such as dedicated PCIe channels) can be allocated to the standardized algorithm modules corresponding to high-priority tasks.

[0045] To address the above steps, standardized algorithm modules with unified input / output data interface formats and communication protocols are selected from the algorithm template library and combined. These modules are then trained using fused data that eliminates domain differences between virtual and real environments. This enables rapid matching, selection, and reuse of standardized algorithm modules from the algorithm template library, avoiding extensive adaptation work during module integration due to inconsistent interfaces. Furthermore, during the matching process, specific standardized algorithm modules can be flexibly replaced or added based on the characteristics of the target scenario, thereby improving the cross-scenario adaptability of the combined standardized algorithm modules.

[0046] S130: Deploy the trained standardized algorithm module to the android to drive the android to perform the target task.

[0047] In this step, a one-click deployment tool can be used to perform model format conversion and environment configuration, and deploy the trained standardized algorithm module to embodied robots in smart healthcare or fintech fields to perform corresponding target tasks, thereby driving and controlling the embodied robots to perform the corresponding target tasks.

[0048] As can be seen, in the above solution, for robots used in surgical assistance in smart healthcare scenarios or intelligent customer service robots in financial services, simulation data and real-world data from multiple scenarios can be collected and mapped to generate fused data with mapping metadata. This eliminates the domain differences between virtual and real environments. Furthermore, based on the type and / or scenario of the embodied robot's target task, corresponding standardized algorithm modules can be selected and combined from an algorithm template library containing standardized algorithm modules categorized by function type and / or task scenario for performing different tasks. The combined standardized algorithm modules are then trained using the fused data with mapping metadata for the corresponding scenario. This avoids extensive adaptation and integration work caused by inconsistent interfaces and data protocols, effectively shortening the integration process. Moreover, training with fused data improves the module's robustness to changes in environmental scenarios, preventing performance degradation after migration to the embodied robot. It also eliminates the need for re-collecting large amounts of new scenario data for repeated training, simplifying the overall control process and improving the control efficiency and accuracy of the embodied robot.

[0049] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0050] Please see Figure 7 As shown, Figure 7 A flowchart illustrating the control method for the embodied robot provided in the second embodiment of the present invention includes the following steps: S210: Collect simulation data and real data from multiple scenarios, establish a mapping relationship between the simulation data and the real data in each scenario, and generate fused data with mapping relationship metadata.

[0051] This step is similar to step S110, and will not be described again here.

[0052] S220: In response to the target execution task of the embodied robot, according to the type and / or task scenario of the target execution task, a standardized algorithm module is selected from the algorithm template library and combined, and the combined standardized algorithm module is trained using the fused data.

[0053] This step is similar to step S120, and will not be described again here.

[0054] S230: Deploy the trained standardized algorithm module to the android to drive the android to perform the target execution task.

[0055] This step is similar to step S130, and will not be described again here.

[0056] S240: Collect the actual operating data of the deployed standardized algorithm module in the operating environment of the embodied robot, and the test operating data of the standardized algorithm module in the simulation environment.

[0057] In this step, real-time operational data of the standardized algorithm module is collected in the operating environment of the embodied robot, and test operational data is collected automatically by running the standardized algorithm module in the simulation environment through preset test cases.

[0058] S250: Perform multi-dimensional performance evaluation based on the real running data and the test running data, generate evaluation results, and identify bottlenecks where performance indicators exceed the corresponding preset thresholds.

[0059] In this step, quantitative analysis is performed based on the collected real and test running data. The real and test running data are evaluated from multiple dimensions, including functional indicators (such as recognition accuracy, action execution precision, and task completion rate), performance indicators (such as inference latency, resource utilization, and energy consumption), and adaptation indicators (such as cross-environment migration accuracy and hardware compatibility). Visual evaluation results are generated. During the performance evaluation process, when a certain indicator is found to exceed the corresponding preset threshold, it is marked as a bottleneck point.

[0060] like Figure 8 As shown, the bottleneck point where the recognition performance index in step S250 exceeds the corresponding preset threshold specifically includes the following steps S251-S252: S251: Compare the measured values ​​of each performance indicator in the evaluation results with the corresponding preset thresholds, and mark the performance indicators that exceed the preset thresholds.

[0061] In this step, the measured values ​​of each performance indicator are compared with preset thresholds one by one. The preset thresholds can be set according to task requirements. For example, the recognition accuracy of the standardized algorithm module used for lesion detection can be set to be no less than 95%, and the resource utilization rate can be set to be no more than 80%. When the measured value of a certain indicator exceeds its preset threshold, it is marked as an over-threshold indicator, and the magnitude and duration of the over-threshold are recorded.

[0062] S252: Based on the data dependency relationship of the standardization algorithm module, associate the marked performance indicators with the corresponding standardization algorithm module to locate the bottleneck point.

[0063] In this step, when a certain indicator is marked as exceeding the threshold, the performance of that indicator in the standardized algorithm module and its preceding standardized algorithm module is checked based on data dependencies to pinpoint the specific location of the bottleneck. For example, when a financial intelligent customer service performs a referral task, if the error rate of the standardized algorithm module used for speech recognition increases, the signal quality of this module and the preceding standardized algorithm module used for audio acquisition will be checked based on data dependencies to determine whether the increased speech recognition error rate is due to signal quality issues in the preceding standardized algorithm module used for audio acquisition (such as too low a sampling rate, too high a signal gain, packet loss, etc.), thereby accurately locating the bottleneck.

[0064] S260: Based on the magnitude and duration of the bottleneck exceeding the corresponding preset threshold, generate optimization parameters, and synchronize the optimization parameters to at least one step before deploying the trained standardized algorithm module to the embodied robot.

[0065] In this embodiment, a supervised learning-based optimization parameter generation model can be used to generate optimization parameters based on the magnitude and duration of the bottleneck exceeding a corresponding preset threshold. Specifically, the supervised learning-based optimization parameter generation model is typically a regression / classification model such as a multilayer perceptron (MLP), random forest, or gradient boosting tree. Training samples include historical bottleneck features (magnitude, duration, and location of the bottleneck exceeding the preset threshold) and their corresponding optimal optimization parameters (such as reducing model compression ratio or increasing training data). The training objective is to minimize the loss function between the model-predicted optimization parameters and the historical optimal optimization parameters, where the optimal optimization parameters can be obtained through expert experience annotation or offline reinforcement learning search. After training, the model can output optimization parameters based on real-time bottleneck features and synchronize the optimization parameters to the aforementioned corresponding steps. For example, if the optimization parameter is to increase the amount of low-light simulation data collected, then the optimization parameter is synchronized to step S210.

[0066] For steps S250-S260 above, the standardized algorithm module in the embodied robot can continuously correct its own configuration during the control operation, forming a continuously evolving closed loop. This can avoid excessive and redundant global retraining, shorten the bottleneck repair cycle, improve performance, and further enhance control accuracy.

[0067] As can be seen, in the above solution, for embodied robots in smart healthcare or financial business scenarios, simulation data and real-world data from multiple scenarios can be collected and mapped to generate fused data with mapping metadata. This eliminates the domain differences between the virtual and real environments. Simultaneously, based on the target task, corresponding standardized algorithm modules can be selected from the algorithm template library and combined. The generated fused data with mapping metadata is then used to train the combined standardized algorithm modules, enabling rapid matching and selection of algorithm modules for collaboratively executing the target task. This avoids extensive adaptation and integration work caused by inconsistent interfaces and data protocols, effectively shortening the integration chain. Furthermore, training with fused data ensures stable performance of the trained algorithm modules during cross-domain migration, avoiding repeated collection and training due to the separation of virtual and real data. After deployment to the embodied robot, there is no need to collect large amounts of new scenario data for repeated training; direct drive control is possible, simplifying the overall control process and improving control efficiency and accuracy. Moreover, a continuously evolving closed loop can be formed during control operation, further enhancing control accuracy.

[0068] In one embodiment, a control device for a android is provided, which corresponds one-to-one with the control method for the android in the second embodiment described above. For example... Figure 9As shown, the control device of the embodied robot includes a data acquisition and processing module 301, a selection and training module 302, a deployment control module 303, and an evaluation and optimization module 304. Detailed descriptions of each functional module are as follows: The data acquisition and processing module 301 is used to acquire simulation data and real data in multiple scenarios, establish the mapping relationship between the simulation data and the real data in each scenario, and generate fused data with mapping relationship metadata. A training module 302 is selected to respond to the target execution task of the embodied robot. Based on the type and / or task scenario of the target execution task, a standardized algorithm module is selected from the algorithm template library and combined. The combined standardized algorithm module is trained using the fused data. The algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario. Deployment control module 303 is used to deploy the trained standardized algorithm module to the embodied robot to drive the embodied robot to perform the target execution task; The data acquisition and processing module 301 is also used to acquire the real operating data of the deployed standardized algorithm module in the operating environment of the embodied robot, as well as the test operating data of the standardized algorithm module in the simulation environment. The evaluation and optimization module 304 is used to perform multi-dimensional performance evaluation based on the real running data and the test running data, generate evaluation results, and identify bottlenecks where performance indicators exceed the corresponding preset thresholds; based on the magnitude and duration of the bottlenecks exceeding the corresponding preset thresholds, it generates optimization parameters and synchronizes the optimization parameters to at least one of the data acquisition and processing module 301 and the training selection module 302.

[0069] In one embodiment, the data acquisition and processing module 301 is specifically used for: The simulation data and the real data are aligned at multiple levels to establish spatial, temporal, and feature correspondences between the simulation data and the real data. A mapping table is constructed using task scenarios and task functions as indexes. This mapping table is used to associate simulation data, real-world data, and their corresponding relationships.

[0070] In one embodiment, the data acquisition and processing module 301 is further specifically used for: Align the simulation data and the real data along the timeline; The spatial co-occurrence features of the time-axis aligned simulation data and real data are extracted, and based on the spatial co-occurrence features, a feature matching algorithm is used to perform spatial geometric alignment of the simulation data and real data; Align the feature distributions of the simulation data and the real data after spatial geometric alignment.

[0071] In one embodiment, the control device for the embodied robot further includes an algorithm library construction module 305, wherein, The data acquisition and processing module 301 is also used to acquire the task execution flow of historical task scenarios, and to break down the task execution flow of each historical task scenario to obtain multiple sub-tasks. The algorithm library construction module 305 is used to cluster the subtasks, merge subtasks with the same or similar functions into the same class, and encapsulate the algorithm flow corresponding to each class of subtasks into a standardized algorithm module and store it in the algorithm template library.

[0072] In one embodiment, the selected training module 302 is further configured to: Based on the task function type and / or task scenario of the target execution task, determine the sub-tasks required to complete the target execution task, and select standardized algorithm modules corresponding to each sub-task from the algorithm template library; Determine the data dependencies and scheduling priorities among the selected standardized algorithm modules to form... The standardized algorithm module for combination.

[0073] In one embodiment, the evaluation and optimization module 304 is specifically used for: The measured values ​​of each performance indicator in the evaluation results are compared with the corresponding preset thresholds, and the performance indicators that exceed the preset thresholds are marked. Based on the data dependencies of the standardized algorithm modules, the performance metrics of the tags are associated with the corresponding standardized algorithm modules to locate the bottleneck points.

[0074] This invention provides a control device for an embodied robot. By collecting simulation data and real-world data from multiple scenarios and establishing a mapping relationship, it generates fused data with mapping relationship metadata to eliminate domain differences between virtual and real environments. Simultaneously, based on the target task, it selects and combines corresponding standardized algorithm modules from an algorithm template library that stores standardized algorithm modules for performing different tasks and categorized by functional type and / or task scenario. The generated fused data with mapping relationship metadata is then used to train the combined standardized algorithm modules. This achieves rapid matching and selection of algorithm modules for collaboratively executing the target task, avoiding extensive adaptation and integration work caused by inconsistent interfaces and data protocols, effectively shortening the integration chain. Furthermore, training with fused data improves the robustness of modules to changes in environmental scenarios, thus preventing performance degradation after migration to the embodied robot and eliminating the need for repeated training with large amounts of new scenario data. This simplifies the overall control process and improves control efficiency and accuracy. Moreover, it can form a continuously evolving closed loop during control operation, further enhancing control accuracy.

[0075] Specific limitations regarding the control device for the embodied robot can be found in the limitations on the control method for the embodied robot described above, and will not be repeated here. Each module in the aforementioned control device for the embodied robot can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0076] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a control method for an embodied robot on the server side.

[0077] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a control method for an embodied robot.

[0078] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Collect simulation data and real data from multiple scenarios, establish mapping relationships between the simulation data and real data in each scenario, and generate fused data with mapping relationship metadata; In response to the target execution task of the embodied robot, standardized algorithm modules are selected from the algorithm template library and combined according to the type and / or task scenario of the target execution task, and the combined standardized algorithm modules are trained using the fused data; wherein, the algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario; The trained standardized algorithm module is deployed to the android to drive the android to perform the target task.

[0079] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Collect simulation data and real data from multiple scenarios, establish mapping relationships between the simulation data and real data in each scenario, and generate fused data with mapping relationship metadata; In response to the target execution task of the embodied robot, standardized algorithm modules are selected from the algorithm template library and combined according to the type and / or task scenario of the target execution task, and the combined standardized algorithm modules are trained using the fused data; wherein, the algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario; The trained standardized algorithm module is deployed to the android to drive the android to perform the target task.

[0080] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0081] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0083] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0084] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A control method for an embodied robot, characterized in that, include: Collect simulation data and real data from multiple scenarios, establish mapping relationships between the simulation data and real data in each scenario, and generate fused data with mapping relationship metadata; In response to the target execution task of the embodied robot, standardized algorithm modules are selected from the algorithm template library and combined according to the type and / or task scenario of the target execution task, and the combined standardized algorithm modules are trained using the fused data; wherein, the algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario; The trained standardized algorithm module is deployed to the android to drive the android to perform the target task.

2. The control method for the embodied robot as described in claim 1, characterized in that, The process of establishing the mapping relationship between the simulation data and the real data in each scenario includes: The simulation data and the real data are aligned at multiple levels to establish spatial, temporal, and feature correspondences between the simulation data and the real data. A mapping table is constructed using task scenarios and task functions as indexes. This mapping table is used to associate simulation data, real-world data, and their corresponding relationships.

3. The control method for the embodied robot as described in claim 2, characterized in that, The multi-level alignment of the simulation data with the real data includes: Align the simulation data and the real data along the timeline; The spatial co-occurrence features of the time-axis aligned simulation data and real data are extracted, and based on the spatial co-occurrence features, a feature matching algorithm is used to perform spatial geometric alignment of the simulation data and real data; Align the feature distributions of the simulation data and the real data after spatial geometric alignment.

4. The control method for the embodied robot as described in claim 1, characterized in that, The control method for the android further includes: Obtain the task execution flow of historical task scenarios, and break down the task execution flow of each historical task scenario to obtain multiple sub-tasks; Cluster the subtasks and merge subtasks with the same or similar functions into the same category; The algorithm flow corresponding to each type of subtask is encapsulated into a standardized algorithm module and stored in the algorithm template library.

5. The control method for the embodied robot as described in claim 1 or 4, characterized in that, The step of selecting and combining standardized algorithm modules from the algorithm template library based on the type and / or task scenario of the target task includes: Based on the task function type and / or task scenario of the target execution task, determine the sub-tasks required to complete the target execution task, and select standardized algorithm modules corresponding to each sub-task from the algorithm template library; Determine the data dependencies and scheduling priorities among the selected standardized algorithm modules to form... The standardized algorithm module for combination.

6. The control method for the embodied robot as described in claim 1, characterized in that, After deploying the trained, standardized algorithm module to the android, the process further includes: The standardized algorithm module was deployed and its actual operation data in the operating environment of the embodied robot was collected, as well as the test operation data of the standardized algorithm module in the simulation environment. Based on the real running data and the test running data, a multi-dimensional performance evaluation is performed to generate evaluation results and identify bottlenecks where performance indicators exceed the corresponding preset thresholds. Based on the magnitude and duration of the bottleneck exceeding the corresponding preset threshold, optimization parameters are generated, and these optimization parameters are synchronized to at least one step before the trained standardized algorithm module is deployed to the embodied robot.

7. The control method for the embodied robot as described in claim 6, characterized in that, The bottleneck points where the recognition performance index exceeds the corresponding preset threshold include: The measured values ​​of each performance indicator in the evaluation results are compared with the corresponding preset thresholds, and the performance indicators that exceed the preset thresholds are marked. Based on the data dependencies of the standardized algorithm modules, the performance metrics of the tags are associated with the corresponding standardized algorithm modules to locate the bottleneck points.

8. A control device for a android, characterized in that, include: The data acquisition and processing module is used to acquire simulation data and real data in multiple scenarios, establish the mapping relationship between the simulation data and the real data in each scenario, and generate fused data with mapping relationship metadata. A training module is selected to respond to the target execution task of the embodied robot. Based on the type of the target execution task and / or the task scenario, standardized algorithm modules are selected from the algorithm template library and combined. The combined standardized algorithm modules are trained using the fused data. The algorithm template library stores standardized algorithm modules for performing different tasks and classified according to functional type and / or task scenario. A deployment control module is used to deploy the trained standardized algorithm module to the embodied robot to drive the embodied robot to perform the target execution task.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the control method for the android as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the control method for the embodied robot as described in any one of claims 1 to 7.

Images