Data processing method, apparatus, device, storage medium, and program product

Abstract

Embodiments of this disclosure relate to a data processing method, apparatus, device, storage medium, and program product. The method includes: receiving first coded data associated with a robot system, the first coded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; verifying the feasibility of the at least one candidate strategy using the first semantic data decoded from the first coded data through a digital twin model, to generate second coded data, the second coded data including second semantic data indicating the feasibility of the at least one candidate strategy, the digital twin model corresponding to the robot system; and sending the second coded data to the robot system to determine an execution strategy associated with the robot system and the at least one task. In this manner, embodiments of this disclosure can effectively ensure the verification efficiency of candidate strategies and the reliability of verification results.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to data processing methods, apparatuses, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] With the rapid development of computer technology, the application of embodied intelligence (Physical AI) systems is becoming increasingly widespread, such as using embodied AI robots to perform tasks like handling, processing, and installation. To ensure the smoothness and safety of the robot system in performing tasks, after generating candidate strategies for the task, twin simulation technology can be used to verify or evaluate these strategies. Then, based on the evaluation results, it is determined whether the robot system should execute the corresponding candidate strategy. In this process, the accuracy and reliability of the evaluation results from twin simulation technology are particularly important. Summary of the Invention

[0003] In a first aspect of this disclosure, a data processing method is provided. The method includes: receiving first encoded data associated with a robot system, the first encoded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; verifying the feasibility of the at least one candidate strategy using the first semantic data decoded from the first encoded data through a digital twin model to generate second encoded data, the second encoded data including second semantic data indicating the feasibility of the at least one candidate strategy, the digital twin model corresponding to the robot system; and sending the second encoded data to the robot system to determine an execution strategy of the robot system associated with the at least one task.

[0004] In a second aspect of this disclosure, a data processing apparatus is provided. The apparatus includes a data receiving module, a data generating module, and a data sending module. The data receiving module is configured to receive first encoded data associated with a robot system, the first encoded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; the data generating module is configured to verify the feasibility of the at least one candidate strategy using the first semantic data decoded from the first encoded data through a digital twin model to generate second encoded data, the second encoded data including second semantic data indicating the feasibility of the at least one candidate strategy, the digital twin model corresponding to the robot system; and the data sending module is configured to send the second encoded data to the robot system to determine an execution strategy of the robot system associated with the at least one task.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can be executed by a processing unit to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] The solution provided in this disclosure analyzes the first encoded data associated with the robot system through a digital twin model to obtain first semantic data corresponding to at least one candidate strategy. Then, the first semantic data is used to verify the feasibility of at least one candidate strategy, thereby effectively ensuring the accuracy and reliability of the feasibility verification results for at least one candidate strategy, ensuring the accuracy and reliability of the second encoded data generated based on the feasibility verification results, and further ensuring the accuracy and reliability of the execution strategy determined based on the second encoded data, thus ensuring the safety of the robot system operation.

[0009] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram is shown of an example environment in which embodiments of the present disclosure may be implemented; Figure 2 A flowchart illustrating an example process of data processing according to some embodiments of this disclosure is shown; Figure 3 A schematic timing diagram of a data processing procedure according to some embodiments of the present disclosure is shown; Figure 4 A schematic structural block diagram of a data processing apparatus according to some embodiments of the present disclosure is shown; Figure 5 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0011] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0012] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0013] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0014] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0015] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0016] As mentioned above, to ensure the smoothness and safety of the robot system in performing its tasks, after generating candidate strategies for the task, twin simulation technology can be used to verify or evaluate these strategies. Then, based on the evaluation results, it can be determined whether the robot system should execute the corresponding candidate strategy. In this process, the accuracy and reliability of the evaluation results from twin simulation technology are particularly important.

[0017] In engineering practice, a common approach is to directly verify the candidate strategies output by the robot system using twin simulation, or to reconstruct the latent representation of the candidate strategy into an explicit state through a temporary interface before performing twin simulation. However, the candidate strategies output by the robot system are often in the form of latent representations, predicted trajectories, or prior evaluation elements, which only represent "action / policy." Twin simulation evaluation, on the other hand, relies on explicit geometric models, physical constraints, and simulators, requiring information such as "action / policy + reproducible evaluation context + traceable constraint semantics." This results in a lack of unified interoperability semantics and consistency mechanisms between the robot system's output and the twin evaluation system, causing the twin evaluation conclusions to drift with contextual differences.

[0018] Specifically, digital twin evaluation technology relies on key contexts such as initial state, object attributes (mass, friction, accessibility, etc.), reference frame, control frequency / time step, and contact and constraint activation conditions. However, the robot system only provides the strategy or potential representation. The twin evaluation system needs to supplement the aforementioned context information through default parameters, historical caching, or external reconstruction. This means that supplementation is required in each evaluation process, resulting in inconsistencies in the context information provided in different batches of evaluation inputs. This inconsistency is not reproducible, leading to different evaluation conclusions for the same candidate strategy at different nodes, versions, or time windows. It is difficult to provide consistent and interpretable safety guarantees.

[0019] In practical applications, the evaluation results provided by twin evaluation systems mostly only include whether it is feasible or not or a single risk score, but cannot obtain key information such as violation type, anomaly occurrence range, related objects and severity. It is difficult to convert the evaluation results into executable screening strategies, such as candidate pruning, rearrangement and search space / constraint boundary adjustment, which leads to the evaluation closed loop degenerating into inefficient trial and error, manifested as strategy jitter, slow convergence or frequent failures in boundary scenarios.

[0020] Robotic systems can generate multiple candidate strategies for multiple tasks in parallel, while twin verification systems may return verification results for different tasks asynchronously. During cross-process or task deployments corresponding to different structures (e.g., center-edge), robotic systems struggle to accurately match verification results for multiple concurrently generated tasks, leading to issues such as mismatches, consumption of expired feedback, duplicate transmissions or executions, introducing security risks, and making the results untraceable.

[0021] Currently, there is an urgent need for a data processing scheme that can decouple robot systems and digital twin systems. This scheme aims to express the minimum necessary semantics and key evaluation context required for twin evaluation in a low-overhead manner, avoiding full reconstruction and redundant transmission while ensuring the reproducibility of evaluation inputs. It should also be able to transmit evaluation conclusions and violation information back in a structured and operable semantic manner, supporting robot systems to perform candidate pruning, sorting, and search space adjustment based on the transmitted information. Furthermore, it should be able to achieve traceable, synchronous, and maintainable bidirectional transmission, thereby achieving efficient interoperability and stable closed-loop collaboration without requiring complete environment reconstruction.

[0022] Embodiments of this disclosure propose a data processing scheme for a robot system. The scheme includes: receiving first coded data associated with the robot system, the first coded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; verifying the feasibility of the at least one candidate strategy using the first semantic data decoded from the first coded data through a digital twin model to generate second coded data, the second coded data including second semantic data indicating the feasibility of the at least one candidate strategy, the digital twin model corresponding to the robot system; and sending the second coded data to the robot system to determine the execution strategy of the robot system associated with the at least one task.

[0023] In this manner, embodiments of the present disclosure parse the first encoded data associated with the robot system through a digital twin model to obtain first semantic data corresponding to at least one candidate strategy. Then, the first semantic data is used to verify the feasibility of at least one candidate strategy, thereby effectively ensuring the accuracy and reliability of the feasibility verification results for at least one candidate strategy, ensuring the accuracy and reliability of the second encoded data generated based on the feasibility verification results, and further ensuring the accuracy and reliability of the execution strategy determined based on the second encoded data, thus ensuring the safety of the robot system operation.

[0024] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0025] Example environment: Figure 1A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include robot system 110 and computing device 120.

[0026] In some embodiments, the robot system 110 may include a robot. The robot system 110 may, for example, be capable of performing tasks such as work or movement through programming and automatic control. As an example, the robot can be implemented in various forms, such as a humanoid robot, a robotic arm, a wheeled robot, etc.

[0027] In some embodiments, the robotic system 110 may include an intelligent system that performs tasks semi-autonomously or fully autonomously through perception, decision-making, and execution. In some embodiments, the robotic system 110 may include an embodied intelligent robot. As an example, an embodied intelligent robot may be a system that possesses a physical body (e.g., motors, robotic arms, sensors, end effectors, etc.) and is capable of perceiving, learning, and intelligently interacting with the real world. For example, the end effector may include structures such as grippers, suction cups, and robotic hands; sensors may include force / torque sensors, vision sensors, position encoders, etc.

[0028] In some embodiments, the robot system 110 may generate at least one candidate strategy 115 based on the task to be performed, and determine the sequence of execution actions corresponding to the task to be performed based on the candidate strategy 115.

[0029] In some embodiments, the robot system 110 may send at least one generated candidate strategy 115 to the computing device 120, whereby the computing device 120 verifies the at least one candidate strategy 115 using a digital twin model 125 and feeds back the verification result to the robot system 110. Then, based on the received feedback result and the at least one candidate strategy 115, the robot system 110 determines the corresponding sequence of execution actions.

[0030] Digital twin model 125 refers to a dynamic virtual model that uses technologies such as geometric modeling, physical law modeling, and behavioral simulation modeling to perform full-element, multi-dimensional digital virtual mapping of physical entities (such as the embodied intelligent robot corresponding to robot system 110) and physical interaction scenarios. It integrates perception data, simulation algorithms, and analysis capabilities, and has the ability to reproduce states, predict behaviors, evaluate performance, and verify strategies. It is the core digital carrier that connects the physical world and the digital world and realizes virtual-real interaction and collaborative optimization.

[0031] In this disclosure, the digital twin model 125 is a virtual mapping model constructed in digital space with explicit geometric features and physical constraint rules. It integrates simulation and evaluation engines and can complete the executability verification and risk quantification assessment of the candidate strategies based on the input candidate strategies and the corresponding evaluation context, and output structured evaluation results. It is the core carrier for the digital twin system to realize physical scene simulation and strategy effectiveness assessment.

[0032] As an example, such a digital twin model 125 can be a virtual mapping model built on the robot system 110, for example, the data structure and configuration logic of the digital twin model 125 are consistent with the data structure and configuration logic of the robot system 110.

[0033] In some embodiments, the computing device 120 may be a standalone hardware device or a software or hardware unit embedded within a hardware device. As an example, such an electronic device may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device may also support any type of user-facing interface (such as "wearable" circuitry).

[0034] It should be understood that the structure and function of the various elements in the example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0035] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0036] Example process: Figure 2 A flowchart of an example process 200 for data processing of a robotic system according to some embodiments of the present disclosure is shown. Example process 200 can be implemented at computing device 120. Reference is made below. Figure 1 Let's describe example process 200.

[0037] like Figure 2As shown, in step 210, computing device 120 receives first encoded data associated with robot system 110. The first encoded data includes first semantic data corresponding to at least one candidate strategy, and the at least one candidate strategy is associated with at least one task of robot system 110.

[0038] In some implementations of this disclosure, the first encoded data may be generated by the robot system 110 and sent to the computing device 120.

[0039] In some embodiments, in response to determining at least one task to be performed, the robot system 110 may determine at least one candidate strategy based on the associated information of the at least one task. As an example, the associated information of the at least one task may include, but is not limited to: task name or sequence number, task object, execution structure, execution target and other task description information, task indicator information, task environment information, etc.

[0040] In some implementations, at least one candidate policy may include a candidate policy generated by the robot system 110 using a task reasoning model. As an example, the robot system 110 may provide task-related information to the task reasoning model to generate and output at least one candidate policy.

[0041] In some implementations, at least one candidate strategy may also include candidate strategies matched by the robot system 110 from a preset strategy set. As an example, the robot system 110 may search for candidate strategies from the preset candidate strategy set that match at least one piece of information related to the task. For instance, the robot system 110 may calculate the matching degree between each candidate strategy in the candidate strategy set and the task based on at least one piece of information related to the task, and then filter at least one candidate strategy from the candidate strategy set according to a preset number or matching degree threshold.

[0042] As an example, the at least one candidate policy determined by the robot system 110 may correspond to a single task or multiple tasks to be executed. In some implementations, the robot system 110 may generate one or more candidate policies for a single task to be executed. In some implementations, the robot system 110 may generate multiple candidate policies in parallel for multiple tasks to be executed, wherein the number of candidate policies is greater than or equal to the number of tasks to be executed.

[0043] After determining at least one candidate strategy, the robot system 110 can encode the at least one candidate strategy and its associated data to generate first encoded data, and send the first encoded data to the computing device 120 to verify the at least one candidate strategy using digital twin technology.

[0044] In some embodiments, to reduce transmission redundancy, the robot system 110 can extract first semantic data corresponding to at least one candidate strategy from the associated data of at least one candidate strategy, and then encode the first semantic data to generate first encoded data. As an example, the associated data of at least one candidate strategy may include, but is not limited to: strategy potential representation, strategy trajectory, environmental state information, evaluation elements, strategy identifier, etc.

[0045] In some examples, the first semantic data can indicate semantic information related to the evaluation or verification of at least one candidate policy. As an example, the first semantic data is used to express minimal semantic information related to the evaluation or verification of at least one candidate policy, enabling the computing device 120 to perform targeted verification and evaluation of each candidate policy without requiring a full environment reconstruction.

[0046] In some embodiments of this disclosure, the first semantic data may include, but is not limited to, at least one of the following: constraint data, interaction data, risk data, scenario data, index data, etc.

[0047] In some implementations, constraint data can be used to describe the constraints of at least one candidate strategy. As an example, constraint data may include, but is not limited to, the constraint type, triggering conditions, applicable scope, priority, threshold range, or protection boundary for each candidate strategy.

[0048] In some implementations, interaction data can be used to describe the interaction conditions associated with at least one candidate strategy and the task object. As an example, interaction data may include, but is not limited to: object pairs interacting in the candidate strategy, the contact type between objects, the contact direction or normal category, the friction or adhesion category, contact evidence association, etc.

[0049] In some implementations, risk data can be used to describe the execution risk corresponding to at least one candidate strategy. As an example, risk data may include, but is not limited to: risk source categories, quantitative or intervalized expressions of risk intensity, upper limits of confidence or uncertainty, and labeled data for risk time segments.

[0050] In some implementations, scene data can be used to describe scene information associated with at least one candidate policy and at least one task. The computing device 120 can reconstruct the local scene associated with the candidate policy based on the scene data, thereby avoiding the reconstruction of the entire scene associated with the robot system 110, avoiding redundant transmission, reducing data processing volume, and saving resources. As an example, scene data may include, but is not limited to: object proxy reference information, local scene reference information, or local geometric / physical summary information used for evaluation.

[0051] In some implementations, index data can be used to describe the execution record information of at least one candidate strategy. As an example, index data may include, but is not limited to, index information associated with observation frames, keyframes, object lists, or observation IDs (identifications). Using index data facilitates the tracing and debugging of candidate strategies and their validation results.

[0052] In some implementations, in response to at least one candidate strategy including only one candidate strategy for a single task, the robot system 110 can extract the corresponding first semantic data from the associated data of the candidate strategy and encode it to generate first encoded data.

[0053] In some implementations, the robot system 110 can generate or assign corresponding verification information for each candidate strategy. For example, such verification information can indicate the strategy identifier, strategy constraints, data integrity, etc., of the candidate strategy. As an example, the robot system 110 can use such verification information as part of the first encoded data, combine it with the first semantic data, and perform an encoding process to generate the first encoded data. For instance, such part could correspond to the metadata portion of the first encoded data.

[0054] In some embodiments of this disclosure, the first encoded data may further include metadata corresponding to at least one candidate strategy, the metadata indicating the correspondence between the first semantic data and at least one candidate strategy.

[0055] In some embodiments, the first encoded data may be generated based on the following process: the robot system 110 generates at least one candidate strategy and metadata corresponding to the at least one candidate strategy based on at least one task; extracts first semantic data from the associated data of the at least one candidate strategy; and encodes the metadata and the first semantic data to generate the first encoded data.

[0056] In some embodiments, the metadata includes at least one of the following: identification data, time-series data, consistency data, and verification data. For example, identification data can be used to describe the policy identifier corresponding to at least one candidate policy; time-series data can be used to describe the execution sequence corresponding to at least one candidate policy, such as the execution sequence corresponding to at least one execution action or execution trajectory in each candidate policy, to support out-of-order processing, duplicate suppression, or expired discarding of multiple candidate policies; consistency data can be used to describe the consistency information of the reference system (e.g., coordinate system) and / or task object corresponding to at least one candidate policy, to achieve reference system (e.g., coordinate system) alignment, object identifier mapping, or object consistency verification; verification data is used to describe the data integrity of the first encoded data, to verify the transmission integrity or reliability of the first encoded data.

[0057] The following is an example of the first encoded data in some embodiments of this disclosure: { "meta": { "schema_version":"1.0", / / [Version / Capability Description Information] Protocol Version "capabilities": ["delta_merge", "partial_parse"], / / Capability tags for version / capability descriptions "msg_type": "ENCODED_EVAL_INPUT", / / [Metadata] Message type "message_id": "WM2TWIN_20260305_120000_042", / / [Verification / Tracing] Unique Message ID "candidate_id": "cand-42", / / [Related Identifier Information] Candidate Strategy Identifier "timestamp_ms": 1772712000000, / / [Time Series Information] Timestamp (ms) "seq": 1001, / / [Time Sequence Information] Serial Number "ttl_ms": 50, / / [Timing Information] Validity period (ms) "source_module": "WORLD_MODEL", / / [Tracing Information] Sender Module "target_module": "DIGITAL_TWIN", / / [Tracing Information] Receiver Module "coord_frame_id": "base_link", / / Coordinate system identifier for alignment / consistency information "object_id_map_hash": "sha256:9f2c...a81d", / / [Alignment / Consistency Information] Object consistency check "checksum": "crc32:7a3f2c1b" / / [Checksum Information] Integrity Check }, "payload": { / / ========================= / / Type 1 encoded message payload: Evaluating input semantics / / (Minimum necessary semantics, complete geometric reconstruction is not required) / / ========================= "constraint_triggers": { / / [Constraint Trigger Semantics] Constraint type / triggering condition / threshold range / protection boundary, etc. "collision_clearance_mm": { "min": 2, "target": 5}, / / Safe clearance threshold range "joint_limit_guard": true, / / Joint limit constraint activated "stability_guard": true, / / Stability constraint activated "speed_limit_mps": 0.35 / / Speed ​​limit constraint }, "contact_semantics": { / / [Interaction / Contact Semantics] Interaction object pairs / Contact types / Normal category / Friction category / Evidence association, etc. "predicted_pairs": [ { "a": "gripper", / / Interactive object A (end-end / gripper) "b": "object_7", / / Interactive object B (target object) "contact_type": "pinch", / / Contact type (structured enumeration) "normal_class": "front", / / Contact direction / normal class "adhesion_class": "none" / / Adhesion / attachment category (optional) } ], "friction_class": "medium", / / Friction category (optional discretization representation) "contact_evidence_ref": { "obs_id": 88421} / / Contact evidence association (pointing to the observation index) }, "risk_evidence": { / / [Risk Evidence Semantics] Risk Source / Risk Intensity Range / Confidence Level or Uncertainty Upper Bound / Time Segment Labeling "risk_kind": "collision", / / Risk source category "risk_interval": { "low": 0.12, "high": 0.25}, / / Interval expression of risk intensity "uncertainty_upper": 0.08, / / Upper bound on uncertainty "time_window_s": { "start": 0.6, "end": 0.8}, / / Time segment annotation (high-risk periods) "confidence": 0.85 / / Confidence level (optional) }, "local_scene": { / / [Object and Local Scene Semantics] Object Proxy / Local Geometry or Physical Summary (Avoiding Full Reconstruction) "proxies": [ { "id": "object_7", / / Object ID "proxy_type": "bbox", / / Object proxy type: bbox / convex hull / simplified mesh, etc. "bbox_m": [0.42, 0.10, 0.05, 0.08, 0.08, 0.12] }, { "id": "table", "proxy_type": "plane", "plane": { "n": [0, 0, 1], "d": -0.73} / / Local geometric summary: plane } ], "physics_summary": { "default_mu": 0.6, / / Local physical summary: Default friction coefficient (example) "gravity_mps2": 9.81 / / Summary of physical constants (example) } }, "evidence_index": { / / [Evidence Index Semantics] Associated with observation frames / keyframes / object lists / observation IDs, used for tracing and debugging. "obs_id": 88421, / / Observation frame / snapshot ID "keyframe_ids": [88421, 88422], / / List of keyframes (optional) "object_list_id": "objlist_T0_9f2c" / / Object list version / index (optional) }, "policy_hint": { / / Candidate strategy suggestion (not a required field): facilitates twin selection of evaluator / window "type": "cartesian_path", "duration_s": 2.0, "control_rate_hz": 50, "stages": ["approach", "grasp", "lift"] } } } It should be noted that the above is only an example of the first encoded data, and this disclosure does not limit the specific first encoded data and its specific content.

[0058] The robot system 110 processes and encodes at least one candidate strategy generated according to preset rules to generate first encoded data, and then sends the first encoded data to the computing device 120 to verify the feasibility of at least one candidate strategy, and receives the corresponding verification results, and then determines the execution strategy corresponding to each task based on the verification results.

[0059] In step 220, the computing device 120 uses the first semantic data decoded from the first encoded data through the digital twin model 125 to verify the feasibility of at least one candidate strategy, so as to generate second encoded data, the second encoded data including second semantic data, the second semantic data indicating the feasibility of at least one candidate strategy, and the digital twin model 125 corresponding to the robot system 110.

[0060] After receiving the first encoded data, the computing device 120 can use the digital twin model 125 to decode the first encoded data to determine the first semantic data corresponding to at least one candidate strategy; then, based on the first semantic data, it can verify the feasibility of at least one candidate strategy and encode the verification result as at least part of the second encoded data to feed back to the robot system 110.

[0061] In some embodiments, after decoding the first semantic data, the computing device 120 may first verify the integrity of the first semantic data and its correspondence with at least one candidate strategy, and then verify the partial semantic data corresponding to each candidate strategy to evaluate the feasibility of the candidate strategy.

[0062] In some embodiments of this disclosure, in response to the first encoded data including metadata, the computing device 120 performs a decoding process on the first encoded data using the decoding module in the digital twin model 125 to obtain metadata and first semantic data corresponding to at least one candidate strategy; then, based on the metadata, the first semantic data can be aligned to obtain aligned data. Such aligned data can indicate the grouping result of the first semantic data with at least one candidate strategy. Then, the computing device 120 can verify the feasibility of at least one candidate strategy based on the aligned data to generate second semantic data; and then encode the metadata and second semantic data to generate second encoded data.

[0063] In some implementations, the computing device 120 can use the digital twin model 125 to split the first semantic data according to at least one candidate strategy based on metadata, and perform alignment processing on the split data according to the timing information corresponding to at least one candidate strategy (such as the timing between multiple strategies, the timing between multiple execution actions of a strategy, etc.) to obtain aligned data corresponding to each candidate strategy.

[0064] The computing device 120 can use this alignment data as input data to the evaluation module in the digital twin model 125 to perform the corresponding verification process, verify the feasibility of each candidate strategy, and obtain the verification result for each candidate strategy. The computing device 120 can combine such verification results with metadata, encode them to obtain second encoded data, and feed it back to the robot system 110.

[0065] In some embodiments, such an evaluation module can perform targeted evaluations on the alignment data corresponding to each candidate strategy to verify the feasibility of each candidate strategy. As an example, such targeted evaluation can indicate at least one verification dimension corresponding to the candidate strategy, such as a geometric dimension, constraint dimension, collaborative consistency dimension, etc.

[0066] In some implementations, the computing device 120 can determine the risk factor corresponding to the candidate strategy in at least one dimension based on the alignment data corresponding to each candidate strategy, then determine at least one verification dimension corresponding to the candidate strategy based on such risk factors, and perform the corresponding verification process to verify the feasibility of the candidate strategy.

[0067] In some implementations, after determining the verification result corresponding to at least one candidate strategy, the computing device 120 can generate second semantic data corresponding to the at least one candidate strategy based on such verification result. This second semantic data can express the verification result of the digital twin model 125 on the candidate strategy through operable semantic content, providing operable feedback information corresponding to the candidate strategy in a structured form for direct consumption by the robot system 110, thereby improving the efficiency of determining the task execution strategy.

[0068] In some embodiments of this disclosure, the second semantic data may indicate the feasibility verification result corresponding to at least one candidate strategy. As an example, such feasibility verification result includes: conclusions or suggestions regarding whether the strategy is executable or not, and may also include prompts corresponding to non-executability, such as reasons and cause analysis information, adjustment methods, or adjustment content.

[0069] In some embodiments, the second semantic data includes, but is not limited to, at least one of the following: feasibility identifiers, assessment data, risk data, operational prompt data, etc.

[0070] In some implementations, a feasibility identifier can be used to describe the execution category corresponding to at least one candidate strategy. Such an execution category may include, for example, executable or recommended execution, non-executable, an executable probability value, or an execution recommendation level. As an example, a feasibility identifier may include: a conclusive description of whether a candidate strategy is executable, or category information or an identifier corresponding to executability.

[0071] In some implementations, the evaluation data can be used to describe the risk assessment results corresponding to at least one candidate strategy. As an example, the evaluation data may include, but is not limited to: risk scores, risk levels or risk interval representations and their optional decomposition information.

[0072] In some implementations, risk data can be used to describe the execution risk associated with at least one candidate strategy. As an example, risk data may include, but is not limited to, at least one of the following: violation type, severity, occurrence range, associated objects, and evidence association information.

[0073] In some implementations, the action prompt data can be used to describe adjustment information corresponding to at least one candidate strategy. As an example, the action prompt data may include actionable adjustment information or suggestions for at least one candidate strategy. For instance, such adjustment information may include, but is not limited to: eliminating candidate strategies, adjusting the priority of candidate strategies, adjusting the constraint boundaries or search controls of candidate strategies, providing supplementary evidence prompts associated with candidate strategies, and triggering the regeneration or correction of candidate strategies.

[0074] In response to determining the second semantic data corresponding to at least one candidate strategy, the computing device 120 can further utilize the encoding module in the digital twin model 125 to encode the second semantic data and the metadata corresponding to the at least one candidate strategy, thereby generating second encoded data corresponding to the at least one candidate strategy. In this way, the metadata can be used to effectively ensure the correlation between the second encoded data and the first encoded data, ensuring the correspondence between the second semantic data in the second encoded data and the at least one candidate strategy, thus ensuring the traceability of the at least one candidate strategy and its verification results.

[0075] This scheme uses structured second-encoded data to achieve operable semantic expression, which is used to indicate the evaluation conclusion and subsequent processing prompts corresponding to at least one candidate strategy, enabling the robot system 110 to perform closed-loop update or application of at least one candidate strategy based on the evaluation results.

[0076] In some embodiments, the encapsulation architecture of the second encoded data and the first encoded data may be the same to ensure the accuracy and stability of bidirectional data transmission between the robot system 110 and the computing device 120, and to ensure the consistency and effectiveness of the transmission, verification and execution processes corresponding to the candidate strategies.

[0077] As an example, the metadata in the second encoded data may be consistent with the metadata in the first encoded data. For instance, the metadata in the second encoded data may include at least a portion of the metadata in the first encoded data.

[0078] In some embodiments, in response to the first encoded data corresponding to multiple candidate strategies, the verification process and cycle duration of the computing device 120 for such multiple candidate strategies are not consistent. In some implementations in this scenario, after all multiple candidate strategies have been verified, the computing device 120 can encode the second semantic data corresponding to each candidate strategy by combining all metadata in the first encoded data to generate second encoded data and return it. In some implementations, the computing device 120 can also generate corresponding second encoded data for each candidate strategy that has been verified, or for multiple candidate strategies that have been verified within a preset time interval, based on the local metadata corresponding to the at least one candidate strategy in the complete metadata of the first encoded data, combined with the second semantic data corresponding to the verification result, and return it. Thus, the timeliness of feedback of the verification results corresponding to the candidate strategies can be guaranteed, and the correspondence between the verification results and the candidate strategies can be guaranteed through metadata, thereby ensuring the accuracy of the application of the verification results in the robot system 110.

[0079] The following is an example of second encoded data in some embodiments of this disclosure: { "meta": { "schema_version":"1.0", / / [Version / Capability Description Information] Protocol Version "capabilities": ["partial_parse"], / / Capability tags for version / capability descriptions "msg_type": "ENCODED_EVAL_FEEDBACK", / / [Metadata] Message Type "message_id": "TWIN2WM_20260305_120000_042", / / [Verification / Tracing] Unique Message ID "candidate_id": "cand-42", / / [Related Identifier Information] Candidate Strategy Identifier "timestamp_ms": 1772712000088, / / [Time Series Information] Timestamp (ms) "seq": 2007, / / [Time Sequence Information] Serial Number "ttl_ms": 50, / / [Timing Information] Validity period (ms) "source_module": "DIGITAL_TWIN", / / [Tracing Information] Sender Module "target_module": "WORLD_MODEL", / / [Tracing Information] Receiver Module "coord_frame_id": "base_link", / / [Alignment / Consistency Information] Reference frame identifier "object_id_map_hash": "sha256:9f2c...a81d", / / [Alignment / Consistency Information] Object consistency check "checksum": "crc32:1c9d0e33" / / [Checksum Information] Integrity Check }, "payload": { / / ========================= / / Second type of encoded message payload: evaluation output and actionable feedback semantics / / ========================= "feasibility": { / / [Feasibility Conclusion] Is it feasible / categorization conclusion? "is_feasible": false, "feasibility_class": "NOT_FEASIBLE_COLLISION" / / Feasibility class (example) }, "risk_assessment": { / / [Risk Assessment Information] Risk Score / Level / Interval Expression and Optional Decomposition "risk_score": 0.78, "risk_level": "HIGH", "risk_interval": { "low": 0.70, "high": 0.86}, / / Interval-based risk expression (optional) "risk_components": { "geometry": 0.62, "contact": 0.11, "dynamics": 0.05 } }, "violations": [ { / / [Structured failure / Violation description] Type / Severity / Occurrence range / Related objects / Evidence correlation, etc. "type": "collision", "severity": "HIGH", "time_window_s": { "start": 0.6, "end": 0.8}, "related_objects": ["object_7"], / / Related objects "location_hint": { "frame": "base_link", "xyz_m": [0.45, 0.12, 0.78] }, "evidence_ref": { "obs_id": 88421} / / Evidence association information } ], "action_hints": [ { / / [Operation Prompt Information] Instructs the inference side on how to adjust the candidate set / search process. "op": "REMOVE_CANDIDATE" / / Remove candidates (example) }, { "op": "ADJUST_CONSTRAINT_BOUNDARY", / / Adjust constraint boundaries / search space (example) "params": { "around_object": "object_7", "margin_mm": 10} }, { "op": "TRIGGER_ALTERNATIVE_CANDIDATE", / / Trigger alternative candidate generation / policy correction (example) "params": { "strategy": "change_grasp_pose"} } ], "debug_info": { / / Debugging / Tracing Information: Does not affect the main semantics "engine": "twin_engine_A", "sim_step_count": 240, "notes": "collision during approach stage" } } } It should be noted that the above is only an example of the second encoded data, and this disclosure does not limit the specific second encoded data or its specific content.

[0080] In step 230, computing device 120 sends second encoded data to robot system 110 to determine the execution strategy of robot system 110 associated with at least one task.

[0081] In this manner, embodiments of the present disclosure can parse the first encoded data associated with the robot system through a digital twin model to obtain the first semantic data corresponding to at least one candidate strategy. Then, the first semantic data is used to verify the feasibility of at least one candidate strategy, thereby effectively ensuring the accuracy and reliability of the feasibility verification results for at least one candidate strategy, ensuring the accuracy and reliability of the second encoded data generated based on the feasibility verification results, and further ensuring the accuracy and reliability of the execution strategy determined based on the second encoded data, thus ensuring the safety of the robot system operation.

[0082] In some embodiments of this disclosure, the second encoded data is configured such that: the robot system 110 can obtain second semantic data by decoding the second encoded data; and then, based on the second semantic data and at least one candidate strategy, determine the execution strategy associated with at least one task by the robot system 110.

[0083] In some implementations, in response to receiving second encoded data, the robot system 110 can obtain metadata and second semantic data corresponding to at least one candidate strategy by decoding the second encoded data. Then, based on the metadata, it determines feedback data corresponding to each candidate strategy in the at least one candidate strategy from the second semantic data. Then, based on such feedback data and at least one candidate strategy, it determines the execution strategy associated with at least one task by the robot system 110.

[0084] As an example, the robot system 110 can perform adjustment operations such as elimination, retention, and optimization on candidate strategies among at least one candidate strategy based on the evaluation results or execution suggestions indicated by the feedback data, thereby determining the execution strategy associated with at least one task.

[0085] In some embodiments of this disclosure, determining an execution strategy for a robot system associated with at least one task based on second semantic data and at least one candidate strategy includes: the robot system 110 determining adjustment information associated with the first strategy in response to determining that the second semantic data indicates adjustment of a first strategy among at least one candidate strategy; adjusting the first strategy based on the adjustment information; and determining that the execution strategy includes the adjusted first strategy.

[0086] As an example, in response to first feedback data in the second semantic data corresponding to the first strategy indicating that the first strategy needs adjustment or optimization, such as the execution category described by the feasibility identifier in the first feedback data being: not recommended to execute (or to execute after adjustment, or needs adjustment, etc.), the robot system 110 can determine the corresponding adjustment information from the first feedback data and adjust the first strategy accordingly. Then, the robot system 110 can use the adjusted first strategy as at least part of the execution strategy, for example, as the execution strategy of using the adjusted first strategy as the first task.

[0087] In some embodiments of this disclosure, the adjustment information indicates at least one of the following: deleting the first strategy; reducing the priority of the first strategy; adjusting the constraint information of the first strategy (e.g., constraint boundaries or search space); adding associated data of the first strategy (e.g., evidence data); regenerating the first strategy, etc.

[0088] In this way, by carrying adjustment information corresponding to at least one candidate strategy in the second semantic data, the embodiments of this disclosure can effectively improve the adjustment efficiency and accuracy of at least one candidate strategy, thereby improving the determination efficiency and accuracy of the execution strategy.

[0089] In some embodiments of this disclosure, determining an execution strategy associated with at least one task based on second semantic data and at least one candidate strategy further includes: the robot system 110 responding to the second semantic data instruction to execute a second strategy among the at least one candidate strategy, and determining that the execution strategy includes the second strategy.

[0090] As an example, in response to second semantic data indicating that a second strategy among at least one candidate strategy is executable, the robot system 110 may adopt the second strategy as the execution strategy for the second task corresponding to the second strategy. In some implementations, second feedback data in the second semantic data corresponding to the second strategy may indicate that the second strategy is an executable strategy. For example, if the feasibility identifier in the second feedback data describes an execution category as executable or recommended, the robot system 110 may determine the second strategy as the execution strategy for the second task.

[0091] In this way, the embodiments of this disclosure can quickly determine the execution strategy corresponding to at least one task based on the second semantic data, effectively improving the efficiency of determining the execution strategy and ensuring the execution efficiency and accuracy of the task.

[0092] In some embodiments of this disclosure, process 200 further includes: computing device 120 receiving third encoded data, the third encoded data including third semantic data associated with a first strategy, the third semantic data indicating incremental information associated with adjustment information; using the first encoded data and the third semantic data decoded from the third encoded data through a digital twin model to verify the feasibility of the adjusted first strategy to generate fourth encoded data; and sending the fourth encoded data to robot system 110.

[0093] Taking the first feedback data indicating the adjustment of the first strategy as an example, the robot system 110 can also send the adjusted first strategy to the computing device 120 for verification. If the verification is successful, the first strategy will be used as at least part of the execution strategy and executed.

[0094] In some implementations, the robot system 110 may encode the complete semantic data of the adjusted first strategy in conjunction with metadata to generate third encoded data and send it to the computing device 120 for verification. In some implementations, the robot system 110 may also encode only the updated data of the adjusted first strategy (e.g., the semantic data corresponding to the adjusted data) in conjunction with metadata to generate third encoded data.

[0095] In some embodiments, the robot system 110 may encode the third semantic data and metadata corresponding to the first strategy into third encoded data and send it to the computing device 120. Such third semantic data indicates updated data that adjusts the first strategy based on adjustment information. This updated data, as incremental data for re-verification, can be combined with the original data corresponding to the first strategy in the first encoded data to verify the adjusted first strategy.

[0096] In this way, the scheme can achieve differential updates and verification of the policy, reduce redundant transmission, save resources, and use metadata to ensure the accuracy and effectiveness of the verification of the adjusted first policy.

[0097] In some embodiments, in response to receiving third encoded data, computing device 120 may decode the third encoded data using digital twin model 125 to obtain third semantic data corresponding to the first strategy. Then, based on the third semantic data, the feasibility of the adjusted first strategy is verified.

[0098] In some embodiments, computing device 120 decodes third encoded data using digital twin model 125 to obtain metadata and third semantic data corresponding to the first strategy; then, it searches for first semantic data corresponding to the first strategy based on the metadata; in response to finding the corresponding first semantic data, it can determine the complete semantic data corresponding to the updated first strategy based on the first semantic data and the third semantic data, and verify the feasibility of the adjusted first strategy accordingly. For example, computing device 120 can extract unadjusted semantic data from the first semantic data and combine it with the third semantic data to determine the complete semantic data corresponding to the adjusted first strategy.

[0099] The computing device 120 encodes the semantic data generated by the verification process based on the third semantic data to generate fourth encoded data, and then returns the fourth encoded data to the robot system 110 to determine the corresponding execution strategy.

[0100] Figure 3 A schematic timing diagram of an example process 300 for data processing according to some embodiments of the present disclosure is shown. Figure 3As shown, in step 311, in response to receiving at least one task to be executed, the robot system 110 can generate at least one corresponding candidate strategy.

[0101] As an example, the robot system 110 can utilize a world model reasoning module to generate one or more candidate policies and their associated evaluation elements. Such evaluation elements can be used to guide the verification process of the candidate policies. For example, these include action timing information, geometric constraint information, contact constraint information, constraint violation information, etc., associated with the candidate policies.

[0102] In step 312, the robot system 110 can also generate a corresponding strategy identifier for each candidate strategy.

[0103] In step 313, the robot system 110 extracts metadata and first semantic data corresponding to at least one candidate strategy, evaluation elements and their corresponding strategy identifiers based on the generated at least one candidate strategy.

[0104] In step 314, the robot system 110 uses the first encoding module to encode the metadata and the first semantic data to generate structured first encoded data.

[0105] In step 315, the robot system 110 sends the first encoded data to the computing device 120 to verify at least one candidate strategy.

[0106] In step 321, in response to receiving the first encoded data, the computing device 120 decodes the first encoded data to obtain metadata and first semantic data corresponding to at least one candidate strategy.

[0107] As an example, computing device 120 can call the decoding module in digital twin model 125 to decode the first encoded data.

[0108] In step 322, computing device 120 aligns the first semantic data using metadata to generate aligned data that can be used for verification. As an example, computing device 120 can use the parsing module in digital twin model 125 to align the first semantic data to generate aligned data. The data structure of such aligned data meets the input requirements of the evaluation module and can be directly used as input data for the evaluation module.

[0109] In step 323, computing device 120 performs a verification process based on the alignment data obtained in step 322 to generate corresponding second semantic data. As an example, computing device 120 can directly provide the alignment data to the evaluation module of digital twin model 125 and use the output of the evaluation module as the second semantic data.

[0110] In step 324, computing device 120 encodes the metadata and the second semantic data to obtain the second encoded data.

[0111] In step 325, the computing device 120 returns the second encoded data to the robot system 110.

[0112] Upon receiving the second encoded data, the robot system 110 can continue to execute step 316.

[0113] In step 316, the robot system 110 decodes the second encoded data to obtain the second semantic data.

[0114] In step 317, the robot system 110 determines an execution strategy that matches the task to be executed based on the second semantic data.

[0115] As an example, the robot system 110 can determine whether each candidate strategy is executable based on the second semantic data. For executable candidate strategies, the robot system 110 can directly use them as at least part of the execution strategy. For candidate strategies that need to be adjusted, the robot system 110 can determine the corresponding adjustment information from the second semantic data and adjust the candidate strategy accordingly. Then, the adjusted strategy can be used as at least part of the execution strategy. Alternatively, the adjusted strategy can be re-encoded and sent to the computing device 120 for verification.

[0116] Example devices and equipment: Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 4 A schematic structural block diagram of a data processing apparatus 400 for controlling a robot system according to certain embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in the computing device 120 discussed above. The various modules / components in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0117] like Figure 4As shown, the apparatus 400 may include a data receiving module 410, a data generating module 420, and a data sending module 430. The data receiving module 410 is configured to receive first coded data associated with a robot system, the first coded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; the data generating module 420 is configured to verify the feasibility of at least one candidate strategy using the first semantic data decoded from the first coded data through a digital twin model, to generate second coded data, the second coded data including second semantic data indicating the feasibility of at least one candidate strategy, the digital twin model corresponding to the robot system; and the data sending module 430 is configured to send the second coded data to the robot system to determine the execution strategy of the robot system associated with at least one task.

[0118] In some embodiments, the first encoded data further includes metadata corresponding to at least one candidate strategy, the metadata indicating the correspondence between the first semantic data and the at least one candidate strategy, and the data generation module 420 is configured to: decode the first encoded data through a digital twin model to obtain metadata and first semantic data; align the first semantic data based on the metadata to obtain aligned data; verify the feasibility of at least one candidate strategy based on the aligned data to generate second semantic data; and encode the metadata and second semantic data to generate second encoded data.

[0119] In some embodiments, the first encoded data is generated based on the following process: generating at least one candidate policy and metadata based on at least one task; extracting first semantic data from the associated data of at least one candidate policy; and encoding the metadata and the first semantic data to generate the first encoded data.

[0120] In some embodiments, the metadata includes at least one of the following: identification data, used to describe the policy identifier corresponding to at least one candidate policy; timing data, used to describe the execution timing corresponding to at least one candidate policy; consistency data, used to describe the consistency information of the reference system and / or task object corresponding to at least one candidate policy; and verification data, used to describe the data integrity of the first encoded data.

[0121] In some embodiments, the second encoded data is configured to: the robot system decode the second encoded data to obtain second semantic data; and based on the second semantic data and at least one candidate strategy, determine an execution strategy associated with at least one task for the robot system.

[0122] In some embodiments, determining an execution strategy for a robotic system associated with at least one task based on second semantic data and at least one candidate strategy includes: determining adjustment information associated with the first strategy in response to determining that the second semantic data indicates an adjustment of a first strategy among the at least one candidate strategy; adjusting the first strategy based on the adjustment information; and determining that the execution strategy includes the adjusted first strategy.

[0123] In some embodiments, the adjustment information indicates at least one of the following: deleting the first policy; reducing the priority of the first policy; adjusting the constraint information of the first policy; adding associated data to the first policy; or regenerating the first policy.

[0124] In some embodiments, determining an execution strategy for the robotic system associated with at least one task based on second semantic data and at least one candidate strategy further includes: determining that the execution strategy includes the second strategy in response to an instruction from the second semantic data to execute the second strategy among the at least one candidate strategy.

[0125] In some embodiments, the apparatus 400 further includes a module that performs the following processes: receiving third encoded data, the third encoded data including third semantic data associated with a first strategy, the third semantic data indicating incremental information associated with adjustment information; verifying the feasibility of the adjusted first strategy using the first encoded data and the third semantic data decoded from the third encoded data through a digital twin model to generate fourth encoded data; and sending the fourth encoded data to the robot system.

[0126] In some embodiments, the first semantic data includes at least one of the following: constraint data, used to describe the constraints of at least one candidate strategy; interaction data, used to describe the interaction conditions associated with at least one candidate strategy and the task object; risk data, used to describe the execution risk corresponding to at least one candidate strategy; scenario data, used to describe the scenario information associated with at least one candidate strategy and at least one task; and index data, used to describe the execution record information of at least one candidate strategy.

[0127] In some embodiments, the second semantic data includes at least one of the following: feasibility identifier, used to describe the execution category corresponding to at least one candidate strategy; evaluation data, used to describe the risk assessment result corresponding to at least one candidate strategy; risk data, used to describe the execution risk content corresponding to at least one candidate strategy; and operation prompt data, used to describe the adjustment information corresponding to at least one candidate strategy.

[0128] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, at least one processing unit 510 or processor, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0129] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0130] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0131] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0132] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0133] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0134] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0135] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0136] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0138] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A data processing method, characterized in that, The method includes: Receive first encoded data associated with a robot system, the first encoded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; The feasibility of at least one candidate strategy is verified using the first semantic data decoded from the first encoded data through a digital twin model, to generate second encoded data. The second encoded data includes second semantic data and local metadata corresponding to the at least one verified candidate strategy. The second semantic data indicates the feasibility of the at least one candidate strategy. The digital twin model corresponds to the robot system. The second encoded data is sent to the robot system, which decodes the second encoded data to obtain the local metadata and the second semantic data. Based on the local metadata, feedback data corresponding to each candidate strategy is determined from the second semantic data. Based on the feedback data and the at least one candidate strategy, the robot system determines the execution strategy associated with the at least one task. Wherein, the first encoded data corresponds to multiple candidate strategies, and the first encoded data also includes metadata corresponding to at least one candidate strategy, the metadata indicating the correspondence between the first semantic data and the at least one candidate strategy, and the step of verifying the feasibility of the at least one candidate strategy using the first semantic data decoded from the first encoded data through a digital twin model to generate second encoded data includes: The first encoded data is decoded using the digital twin model to obtain the metadata and the first semantic data; Based on the metadata, the first semantic data is aligned to obtain aligned data; Based on the alignment data, the feasibility of the at least one candidate strategy is verified to generate the second semantic data; and For at least one candidate strategy that has been verified, the local metadata corresponding to the at least one candidate strategy that has been verified is determined from the metadata, and the local metadata and the corresponding second semantic data are encoded to generate the second encoded data.

2. The method according to claim 1, characterized in that, The first encoded data is generated based on the following process: Based on the at least one task, generate the at least one candidate strategy and the metadata; Extract the first semantic data from the associated data of the at least one candidate strategy; as well as The metadata and the first semantic data are encoded to generate the first encoded data.

3. The method according to claim 1, characterized in that, The metadata includes at least one of the following: Identification data, used to describe the strategy identifier corresponding to the at least one candidate strategy; Time-series data is used to describe the execution time sequence corresponding to the at least one candidate strategy; Consistency data is used to describe the consistency information of the reference frame and / or task object corresponding to the at least one candidate strategy; Verification data is used to describe the integrity of the first encoded data.

4. The method according to claim 1, characterized in that, Determining the execution strategy associated with the robot system and the at least one task based on the second semantic data and the at least one candidate strategy includes: In response to determining that the second semantic data indicates an adjustment to the first strategy among the at least one candidate strategy, adjustment information associated with the first strategy is determined; Based on the adjustment information, adjust the first strategy; and The execution strategy is determined to include the adjusted first strategy.

5. The method according to claim 4, characterized in that, The adjustment information indicates at least one of the following: Delete the first strategy; Reduce the priority of the first strategy; Adjust the constraint information of the first strategy; Add associated data for the first strategy; Regenerate the first strategy.

6. The method according to claim 4, characterized in that, The step of determining the execution strategy associated between the robotic system and the at least one task based on the second semantic data and the at least one candidate strategy further includes: In response to the second semantic data indicating that the second strategy among the at least one candidate strategy is executed, it is determined that the execution strategy includes the second strategy.

7. The method according to claim 4, characterized in that, The method further includes: Receive third encoded data, the third encoded data including third semantic data associated with the first strategy, the third semantic data indicating incremental information associated with the adjustment information; The feasibility of the adjusted first strategy is verified using the digital twin model, utilizing the first encoded data and the third semantic data decoded from the third encoded data, to generate fourth encoded data; and The fourth encoded data is sent to the robot system.

8. The method according to claim 1, characterized in that, The first semantic data includes at least one of the following: Constraint data, used to describe the constraints of the at least one candidate strategy; Interaction data, used to describe the interaction conditions associated with the at least one candidate strategy and the task object; Risk data is used to describe the execution risk corresponding to the at least one candidate strategy; Scene data, used to describe scene information associated with the at least one candidate strategy and the at least one task; Index data, used to describe the execution record information of the at least one candidate strategy.

9. The method according to claim 1, characterized in that, The second semantic data includes at least one of the following: A feasibility identifier is used to describe the execution category corresponding to the at least one candidate strategy; Evaluation data is used to describe the risk assessment results corresponding to the at least one candidate strategy; Risk data is used to describe the execution risk content corresponding to the at least one candidate strategy; Operation prompt data is used to describe the adjustment information corresponding to the at least one candidate strategy.

10. A data processing apparatus, characterized in that, The device includes: A data receiving module is configured to receive first encoded data associated with a robot system, the first encoded data including first semantic data corresponding to at least one candidate strategy, the at least one candidate strategy being associated with at least one task of the robot system; A data generation module verifies the feasibility of at least one candidate strategy using the first semantic data decoded from the first encoded data through a digital twin model, thereby generating second encoded data. The second encoded data includes second semantic data and local metadata corresponding to the at least one candidate strategy that has been verified. The second semantic data indicates the feasibility of the at least one candidate strategy. The digital twin model corresponds to the robot system. A data sending module is configured to send the second encoded data to the robot system, so that the robot system decodes the second encoded data to obtain the local metadata and the second semantic data, determines feedback data corresponding to each candidate strategy from the second semantic data based on the local metadata, and determines the execution strategy associated with the robot system and the at least one candidate strategy based on the feedback data and the at least one candidate strategy. Wherein, the first encoded data corresponds to multiple candidate strategies, and the first encoded data also includes metadata corresponding to at least one candidate strategy, the metadata indicating the correspondence between the first semantic data and the at least one candidate strategy, and the data generation module is configured to: The first encoded data is decoded using the digital twin model to obtain the metadata and the first semantic data; Based on the metadata, the first semantic data is aligned to obtain aligned data; Based on the alignment data, the feasibility of the at least one candidate strategy is verified to generate the second semantic data; and For at least one candidate strategy that has been verified, the local metadata corresponding to the at least one candidate strategy that has been verified is determined from the metadata, and the local metadata and the corresponding second semantic data are encoded to generate the second encoded data.

11. An electronic device, characterized in that, The electronic device includes: At least one processing unit; and At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that, The computer-executable instructions can be executed by a processing unit to implement the method according to any one of claims 1 to 9.

13. A computer program product, said computer program product being tangibly stored in a computer storage medium and comprising computer-executable instructions, characterized in that, The computer-executable instructions, when executed by the device, cause the device to perform the method according to any one of claims 1 to 9.

Images