End effector execution method, device and medium

Abstract

The present application provides an end effector execution method, device, and medium, which relate to the field of artificial intelligence technology. A world model is constructed based on environmental information of the end effector performing a target interaction task; based on a first preset model, a target structure of the end effector is determined according to the world model and the task requirements of the end effector, as well as an interactive behavior strategy for the end effector to perform the target interaction task; based on a second preset model, structural manufacturing instructions for the end effector to manufacture the target structure, as well as quality control rules for the end effector to manufacture the target structure, are determined according to the target structure; the end effector is controlled to manufacture the target structure on demand in real time through a multi-process additive manufacturing function; and when the target structure is manufactured, the target effector is controlled to perform the target interaction task according to the interactive behavior strategy, interactive motion control instructions, and feedback compensation mechanism. The present application can improve the performance of the end effector in performing tasks.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an end effector execution method, device, and medium. Background Art

[0002] As a key component for robots to directly perform tasks, the performance and adaptability of the end-effector directly determine the robot's ability to complete these tasks. In unstructured scenarios such as mass-customized factories, home services, and field exploration, end-effectors must cope with complex and changing environments and diverse operational tasks. Currently, end-effector execution methods for unstructured environments struggle to adapt to these complex environments and task requirements. In these unstructured scenarios, complex situations frequently arise, such as the strong coupling of geometric and physical properties of the manipulated objects, rigid-flexible composites, diverse surface characteristics, and temperature and humidity constraints on the contact surfaces. This places stringent demands on the control accuracy and robustness of robotic manipulation strategies. Existing technologies primarily achieve object interaction through customized soft robots that utilize pneumatic or hydraulic pressure to drive the deformation of soft materials. However, these customized structures are designed for specific tasks and environments, lacking universality and adaptability. They are difficult to quickly adjust to varying geometric features and physical properties. This results in difficulty ensuring control accuracy and insufficient robustness of interaction strategies when the end-effector performs tasks, ultimately leading to poor task execution. Summary of the Invention

[0003] The present application provides an end effector execution method, device, and medium, which can improve the execution effect of the end effector in executing tasks.

[0004] In a first aspect, an embodiment of the present application provides an end effector execution method, the method comprising:

[0005] A world model is constructed based on the environmental information of the end effector in performing the target interactive task; the world model is used to characterize the physical and dynamic characteristics of the interactive environment in which the end effector performs the target interactive task;

[0006] Based on the first preset model, the target structure of the end effector and the interactive behavior strategy of the end effector in performing the target interactive task are determined according to the world model and the task requirements of the end effector; the first preset model is trained with the goal of maximizing the execution success rate of the target interactive task;

[0007] Based on the second preset model, structural manufacturing instructions for the end effector to manufacture the target structure and quality control rules for the end effector to manufacture the target structure are determined according to the target structure; and based on the interactive behavior strategy, interactive motion control instructions for the end effector to execute the interactive behavior strategy and a feedback compensation mechanism for the end effector to execute the target interactive task according to the interactive motion control instructions are determined; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task;

[0008] Based on the world model, target structure, structure manufacturing instructions and quality control rules, the end effector is controlled to manufacture the target structure on demand in real time through the multi-process additive manufacturing function;

[0009] When the target structure is manufactured, the target actuator is controlled to perform the target interaction task according to the interactive behavior strategy, interactive motion control instructions and feedback compensation mechanism. The target actuator is the end effector with the target structure.

[0010] In a second aspect, the present application provides an end effector execution device, the device comprising:

[0011] A construction module, configured to construct a world model based on environmental information of the end effector performing the target interactive task; the world model is configured to characterize the physical and dynamic characteristics of the interactive environment in which the end effector performs the target interactive task;

[0012] a first determination module, configured to determine, based on a first preset model and in accordance with the world model and the task requirements of the end effector, a target structure of the end effector and an interactive behavior strategy for the end effector to perform the target interactive task; wherein the first preset model is trained with the goal of maximizing the execution success rate of the target interactive task;

[0013] a second determination module for determining, based on a second preset model and in accordance with the target structure, structural manufacturing instructions for the end effector to manufacture the target structure, as well as quality control rules for the end effector to manufacture the target structure; and, based on the interactive behavior strategy, determining interactive motion control instructions for the end effector to execute the interactive behavior strategy, as well as a feedback compensation mechanism for the end effector to execute the target interactive task in accordance with the interactive motion control instructions; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task;

[0014] a first control module, configured to control the end effector to manufacture the target structure on demand in real time through a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instruction, and the quality control rule;

[0015] The second control module is used to control the target actuator to perform the target interaction task according to the interactive behavior strategy, the interactive motion control instructions and the feedback compensation mechanism when the target structure is manufactured. The target actuator is an end actuator having the target structure.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0017] When the processor executes the computer program instructions, the end effector execution method in any one of the embodiments of the first aspect is implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, an end effector execution method as in any one of the embodiments in the first aspect is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the end effector execution method in any one of the embodiments in the first aspect above.

[0020] In an end-effector execution method, device, and medium provided in embodiments of the present application, a world model is constructed to comprehensively characterize the physical and dynamic characteristics of the interactive environment, accurately reflecting the actual interactive environment. Based on a first preset model trained to maximize task execution success rate, a target structure and interactive behavior strategy are determined in combination with the world model and task requirements. A second preset model, trained to improve structural quality and interactive robustness, determines structural manufacturing instructions, quality control rules, interactive motion control instructions, and a feedback compensation mechanism. Consequently, during the manufacture of the target structure, the quality control rules ensure the reliability of the target structure; thus, the manufactured end-effector with the target structure can effectively improve task execution efficiency and success. The feedback compensation mechanism provides supplementary support during the target effector's execution of the interactive task, making the target effector more stable and adaptable during task execution. It can timely adjust the target effector's interactive actions to maintain the normal progress of the interactive task, significantly improving the robustness of the interactive behavior strategy, avoiding task interruption or failure, and thus enhancing the performance of the end-effector in executing the interactive task. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 This is one of the flow charts of the end effector execution method provided in the embodiment of the present application;

[0023] Figure 2 This is the second flow chart of the end effector execution method provided in the embodiment of the present application;

[0024] Figure 3 This is a structural diagram of an end effector structure case 1 in a simulation environment provided by an embodiment of the present application;

[0025] Figure 4 This is a structural diagram of case 2 of an end effector structure in a simulation environment provided by an embodiment of the present application;

[0026] Figure 5 This is a structural diagram of case 3 of an end effector structure in a simulation environment provided by an embodiment of the present application;

[0027] Figure 6 This is a schematic diagram of three interaction strategies for a sphere in Example 1 of an end effector structure provided in an embodiment of the present application;

[0028] Figure 7 Schematic diagram of two installation and application methods of an end effector structure case 1 provided in an embodiment of the present application;

[0029] Figure 8 This is an interactive application method of an end effector structure case 3 provided in an embodiment of the present application;

[0030] Figure 9 This is a structural diagram of an end effector execution device provided in an embodiment of the present application;

[0031] Figure 10 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0034] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0035] In order to solve the problems existing in the related art, the embodiments of the present application provide an end effector execution method, device and medium.

[0036] The following first introduces the end effector execution method provided in the embodiment of the present application. Figure 1 As shown, the method specifically includes the following steps:

[0037] S100 , constructing a world model based on the environment information of the end effector performing the target interactive task; the world model is used to characterize the physical and dynamic characteristics of the interactive environment in which the end effector performs the target interactive task.

[0038] Optionally, in embodiments of the present application, an end effector is a component on a robot or automated device that directly performs tasks, similar to a human hand or tool, and is a key component for achieving specific operations. The end effector directly interacts with the object being manipulated, for example, in industrial production, for grasping and moving parts, or in household services, for cleaning and picking up items. The performance and design of the end effector directly impact the effectiveness of task execution, and different task requirements require end effectors of different types and functions.

[0039] A target interaction task refers to a specific objective that the end effector needs to complete and serves as the core guide for the entire operation. For example, in a mass customization factory, a target interaction task might involve precisely assembling a specific workpiece; in a home service scenario, it might involve cleaning the floor and organizing items; or in field exploration, it might involve collecting specific samples. These tasks have clear expected outcomes and drive the end effector's subsequent actions and decisions.

[0040] Environmental information refers to various types of information about the environment in which the end effector performs the target interaction task. This information covers a wide range of topics, including information about objects in the interactive environment, such as their position, shape, size, and material; physical conditions such as temperature, humidity, and light intensity; and the spatial layout of the environment, such as the presence of obstacles and the width of passageways. It also includes information about the interactive objects and the end effector itself, such as the physical properties of the objects and the position and posture of the end effector itself. This information serves as the foundational data for building a world model.

[0041] A world model is constructed based on collected environmental information to represent the interactive environment. It is a digital, abstract representation of the real-world interactive environment, integrating various elements and relationships within it. By constructing a world model, the end effector can "understand" its environment, providing a basis for subsequent decision-making. This world model not only contains static information about the environment but also describes its dynamic changes, enabling the end effector to predict environmental evolution and make adaptive decisions in advance.

[0042] The interaction environment is the specific environment within which the end effector performs its target interaction task. It can be a factory floor, a house, or a field. The interaction environment encompasses various objects, spaces, and conditions relevant to the task. It is the place where the end effector interacts with the object being manipulated. Its complexity and characteristics directly affect the difficulty and method of task execution.

[0043] Physical properties primarily refer to the physical properties of objects and the environment itself within the interactive environment. Examples include an object's density, hardness, elastic modulus, and thermal conductivity. These properties determine how the object behaves under stress and heat. Environmental physical properties also include temperature, humidity, and air pressure, which influence the performance of the end effector and its interaction with the object. For example, high temperatures can affect the mechanical properties of end effectors made of certain materials, while humid environments can cause electronic component failures.

[0044] Dynamic characteristics describe the time-varying characteristics of the interactive environment and the objects within it. These include the object's motion state, such as velocity, acceleration, and trajectory; dynamic changes in environmental conditions, such as temperature fluctuations and changes in lighting; and the dynamic response of the end effector during interaction with the object, such as changes in contact force and object deformation. Understanding dynamic characteristics helps the end effector adjust its actions and strategies in a timely manner to adapt to the dynamic demands of the environment and task, ensuring smooth execution.

[0045] Optionally, in a feasible implementation of the present application, in the data collection stage, ultrasonic sensors can be used to assist in detecting distance, and inertial measurement units can be used to obtain end-effector posture information; in terms of data processing, clustering algorithms in machine learning can be used to classify and integrate data, or deep learning autoencoders can be used to automatically extract features; in the model building stage, knowledge graph technology can be introduced to structure various types of knowledge and relationships in the environment to make the world model more semantically understandable, or combined with virtual reality / augmented reality technology, it can assist in building and verifying the world model in a visual way to enhance the intuitiveness and accuracy of the model.

[0046] S200, based on a first preset model, according to the world model and the task requirements of the end effector, determine the target structure of the end effector and the interactive behavior strategy of the end effector to perform the target interactive task; the first preset model is trained with the goal of maximizing the execution success rate of the target interactive task.

[0047] Optionally, in this embodiment of the present application, the first preset model is a model built based on deep learning, trained to maximize the success rate of executing the target interactive task. After training with a large amount of data, it is able to analyze the environmental information provided by the world model and output appropriate decision results based on the task requirements of the end effector.

[0048] Task requirements refer to the specific requirements and expected outcomes that an end-effector must achieve when performing a target interaction task. These requirements encompass the task type (e.g., grasping, assembly, handling), task accuracy requirements, time limits, and specific constraints on the object being manipulated (e.g., gentle handling of fragile items). Task requirements are key to driving the entire end-effector operational process, clarifying the direction of the end-effector's efforts and serving as crucial input for decision-making within the first pre-defined model.

[0049] The target structure is the structural proposal for the end effector, determined by the first pre-defined model based on the world model and task requirements. This structure encompasses aspects of the end effector's shape, size, material distribution, and joint configuration, aiming to meet the kinematic accessibility requirements during task execution and possess appropriate mechanical stiffness and compliance. For example, in precision assembly tasks, the target structure might be designed to be geometrically adaptable to ensure robust operation; whereas, in heavy object handling tasks, the target structure must possess sufficient strength and stability.

[0050] An interactive behavior strategy is a collection of actions and decisions taken by an end-effector to perform a targeted interactive task. It includes a basic action sequence—the specific steps and sequence of actions the end-effector must perform to complete the task, such as approaching an object, grasping it, moving it, and placing it. It also includes a library of safety strategies to address potential risks that may arise during task execution, such as preventing collisions and overloads. This interactive behavior strategy ensures that the end-effector can safely and effectively interact with the object and environment to achieve the task's objectives.

[0051] The execution success rate is a metric that measures the degree to which the end effector successfully executes the target interactive task. It is used to evaluate the effectiveness of the first preset model's decisions. It represents the proportion of times the end effector successfully completes the task when performing the same or similar tasks multiple times. The criteria for successful task completion can be set based on task requirements, such as accurate assembly or handling within a specified timeframe. The first preset model aims to improve this metric through continuous optimization training, enabling the end effector to perform its tasks more reliably in a variety of environments and task conditions.

[0052] Optionally, in a feasible implementation of the present application, the world model and the task requirements of the end effector are input into the first preset model. The first preset model will perform in-depth analysis and processing on the input information. The first preset model will evaluate the feasibility and effectiveness of different structures and behavior strategies in the interactive environment based on the environmental conditions reflected by the world model. For example, if the environmental space is narrow, the first preset model may tend to choose a smaller and more flexible target structure; if the surface of the operating object is smooth, the first preset model will design an interactive behavior strategy that is more suitable for grasping smooth objects. In determining the target structure, the first preset model will comprehensively consider the requirements of the task requirements for the structure, such as load-bearing capacity, operating accuracy, etc., as well as the limiting factors of the environment in the world model, to generate the end effector structure solution that best meets the maximum execution success rate, including the specific design of the shape, size, material, etc. of the structure.

[0053] For the interactive behavior strategy, the first pre-defined model plans a series of action steps and decision rules. These rules cover the entire process from approaching the object, performing the operation, to completing the task. Ultimately, the first pre-defined model outputs the target structure of the end effector and the interactive behavior strategy for executing the target interactive task, providing key information for subsequent steps.

[0054] S300, based on the second preset model, according to the target structure, determine the structural manufacturing instructions for the end effector to manufacture the target structure, and the quality control rules for the end effector to manufacture the target structure; and according to the interactive behavior strategy, determine the interactive motion control instructions for the end effector to execute the interactive behavior strategy, and the feedback compensation mechanism for the end effector to perform the target interactive task according to the interactive motion control instructions; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task.

[0055] Optionally, in an embodiment of the present application, the second preset model takes the target structure and interactive behavior strategy as input, and outputs structural manufacturing instructions, quality control rules, interactive motion control instructions and feedback compensation mechanisms, etc., to provide guidance for the manufacturing and motion control of the end effector.

[0056] Structural manufacturing instructions are a series of instructions determined based on the target structure, which are used to guide the end effector on how to manufacture the target structure. They include specific operating steps, action sequence, process parameters, etc., and clarify the specific behavior of the end effector during the manufacturing process.

[0057] Quality control rules are established to ensure that the target structure meets certain quality standards. These rules involve monitoring and adjusting various parameters during the manufacturing process, such as material selection, machining accuracy control, and manufacturing speed adjustment, to ensure stable and reliable quality of the target structure.

[0058] Interactive motion control instructions are instructions generated according to the interactive behavior strategy, which are used to control the movement mode of the end effector when performing the target interactive task. They include control information such as the movement trajectory, speed, acceleration, and posture, so that the end effector can interact with the environment according to the predetermined interactive behavior strategy.

[0059] Feedback compensation is a mechanism that adjusts and compensates the end-effector's motion based on feedback information. During the execution of a target interactive task, sensors and other devices capture the difference between the actual motion state and the expected motion state. This feedback information is then used to adjust the interactive motion control instructions to correct the deviation and improve the accuracy and stability of the task.

[0060] Structural quality refers to the quality characteristics of the target structure, including strength, stiffness, precision, stability, and other aspects. A high-quality target structure can better meet mission requirements, withstand corresponding external forces and environmental influences, and ensure the reliability and performance of the end effector during mission execution.

[0061] Interaction robustness in executing target interaction tasks refers to the end-effector's ability to withstand various uncertainties and interferences while executing the target interaction task. High interaction robustness means that the end-effector can maintain stable performance and a high success rate despite varying environmental conditions, task changes, and potential interference. This means that the end-effector can flexibly adapt to various situations and accurately complete the interaction task.

[0062] Optionally, in one feasible implementation of the present application, the determined target structure is input into a second preset model. Based on this information, combined with the knowledge and experience acquired through training, the second preset model takes into account various factors in the manufacturing process, such as the feasibility of the processing technology and the performance limitations of the equipment, and generates structural manufacturing instructions for the end effector to manufacture the target structure, guiding the end effector on how to construct the target structure step by step.

[0063] To ensure the quality of the target structure, the second pre-set model also formulates corresponding quality control rules based on the characteristics and requirements of the target structure. This may involve regulations on material quality control during the manufacturing process, precision requirements for key dimensions and shapes, and testing methods and standards during the manufacturing process. By strictly adhering to these quality control rules, the manufactured target structure can be guaranteed to meet predetermined quality standards and exhibit excellent performance and reliability.

[0064] Next, the interactive behavior strategy is input into a second preset model. Based on this information, the second preset model considers various potential scenarios during task execution, such as environmental changes and uncertainty regarding the manipulated object, and determines the interactive motion control instructions for the end effector to execute the interactive behavior strategy. These instructions precisely control the end effector's motion, including speed, acceleration, and posture adjustment, enabling the end effector to interact with the environment accurately according to the interactive behavior strategy.

[0065] Finally, in order to deal with the deviations and interferences that may occur during the execution of interactive motion control instructions, the second preset model will formulate a feedback compensation mechanism based on the interactive behavior strategy and possible error conditions to ensure that the end effector can perform the target interactive task stably and accurately.

[0066] S400, controlling the end effector to manufacture the target structure in real time on demand through a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instruction, and the quality control rule;

[0067] S500, when the target structure is manufactured, controlling the target actuator to perform the target interaction task according to the interactive behavior strategy, the interactive motion control instruction and the feedback compensation mechanism, wherein the target actuator is an end actuator having the target structure.

[0068] Optionally, in an embodiment of the present application, the target actuator refers to an actuator formed by adding a specifically designed target structure to the basic end actuator through the real-time additive manufacturing function. The basic end actuator is an actuator in the initial state without structural manufacturing. The basic end actuator has basic movement and operation functions, but has not yet been structurally optimized for specific interactive tasks. The target structure manufactured in real time through the multi-process additive manufacturing function is a special functional module designed according to task requirements. After combining the target structure with the basic end actuator, a target actuator is formed, so that the target actuator has the kinematic accessibility, stiffness and compliance required for the interactive task in structure, and functionally supports specific interactive behavior strategies, thereby significantly improving the task execution capability.

[0069] Optionally, in a feasible implementation of the present application, structural manufacturing instructions, quality control rules, interactive motion control instructions and feedback compensation mechanisms work together. The structural manufacturing instructions guide the multi-process additive manufacturing module to complete the target structure manufacturing, and the quality control rules ensure that the manufactured structure meets the quality standards. The interactive motion control instructions accurately control the motion trajectory, speed and posture of the end effector based on the interactive behavior strategy. The feedback compensation mechanism monitors the motion state and interaction of the target actuator in real time. Once deviations such as collision and slippage occur, the interactive motion control instructions are immediately adjusted to correct the deviation. Throughout the process, the various elements cooperate with each other to dynamically adjust the actions of the target actuator so that it can accurately complete the target interactive task and improve the success rate of task execution.

[0070] In an end-effector execution method provided in an embodiment of the present application, a world model is constructed to comprehensively characterize the physical and dynamic characteristics of the interactive environment, accurately reflecting the actual interactive environment. Based on a first preset model trained to maximize task execution success rate, the target structure and interactive behavior strategy are determined in combination with the world model and task requirements. A second preset model, trained to improve structural quality and interactive robustness, determines structural manufacturing instructions, quality control rules, interactive motion control instructions, and a feedback compensation mechanism. Consequently, during the manufacture of the target structure, the quality control rules ensure the reliability of the target structure; thus, the manufactured end-effector with the target structure can effectively improve task execution efficiency and success. The feedback compensation mechanism provides supplementary support during the target effector's execution of the interactive task, making the target effector more stable and adaptable during task execution. It can timely adjust the target effector's interactive actions to maintain the normal progress of the interactive task, significantly improving the robustness of the interactive behavior strategy, avoiding task interruption or failure, and thus enhancing the performance of the end-effector in executing the interactive task.

[0071] In one embodiment, constructing a world model based on environmental information of the end effector performing the target interaction task includes:

[0072] Acquiring the environmental information, wherein the environmental information includes geometric information and physical information of the interactive environment, the interactive object of the interactive task, and the actuator body of the end effector;

[0073] Performing time calibration and feature extraction on the environmental information to obtain target feature information;

[0074] The world model is constructed according to the target feature information, and the world model is used to realize dynamic perception prediction of the interactive environment.

[0075] Optionally, in the embodiment of the present application, the interactive object refers to the object with which the end effector directly interacts during the execution of the target interactive task. In different task scenarios, the interactive objects will be different. In industrial manufacturing scenarios, they may be parts to be processed or components to be assembled; in home service scenarios, they may be furniture that needs to be cleaned or items that need to be moved; in field exploration scenarios, they may be plants, geological rocks, etc. that need to be sampled. The characteristics of these interactive objects, such as shape, material, weight, etc., will have an important impact on the operation mode and strategy of the end effector.

[0076] Geometric information primarily describes spatial features of an object, such as its shape, size, position, and posture. For interactive environments, this information can include the outlines and layout of objects within the environment, as well as the size and shape of the space. For interactive objects, this information reflects their specific appearance, including whether they are regular or irregular, and their dimensions. For the end effector itself, geometric information encompasses the shape, size, position, and posture of its components in space. This information influences how the end effector interacts with the object and its reach.

[0077] Physical information encompasses the various physical properties of objects. For the interactive environment, this includes environmental factors such as temperature, humidity, light intensity, and gravity. For the interactive object, this includes material (such as metal, plastic, or wood), density, hardness, elastic modulus, and friction coefficient. These properties determine how the object behaves under stress and heat. For the end effector itself, physical information encompasses mass, inertia, motor torque, and joint stiffness. This information influences the end effector's motion performance and operational capabilities.

[0078] In these optional embodiments, geometric and physical information about the interactive environment, interactive objects, and the end-effector's actuator body is acquired, providing fundamental data for a comprehensive understanding of the interactive environment and enabling the end-effector to perceive its surroundings and its own status. Time calibration and feature extraction further optimize the data, eliminating spatiotemporal deviations between sensor data, extracting key target feature information, and improving data quality and usability. Finally, a world model is constructed based on the target feature information, accurately characterizing the physical and dynamic characteristics of the interactive environment. This provides a reliable basis for the end-effector to subsequently determine target structure, interactive behavior strategies, and other aspects. This enhances the end-effector's adaptability and decision-making accuracy in complex mission scenarios, helping to improve the success rate of mission execution.

[0079] In one embodiment, performing time calibration and feature extraction on the environmental information to obtain target feature information includes:

[0080] Establishing a unified coordinate system for the environmental information collected from different collection channels based on a three-dimensional space registration algorithm to obtain first information;

[0081] Performing time calibration on the first information by using a distributed timestamp combined with a least squares algorithm to obtain calibration information;

[0082] Feature extraction is performed on the calibration information to obtain the target feature information.

[0083] In these optional embodiments, a unified coordinate system is established based on a three-dimensional spatial registration algorithm, and environmental information from different acquisition channels is integrated into the same spatial framework, avoiding information confusion caused by differences in coordinate systems, ensuring the consistency of information spatial positions, and facilitating subsequent comprehensive analysis. Distributed timestamps are combined with a least squares algorithm for time calibration, eliminating the time deviation of information, synchronizing each information in the time dimension, and ensuring data timeliness and consistency. Feature extraction of calibration information can extract key target feature information from complex data, remove redundancy, highlight useful information, and provide strong support for building an accurate world model, thereby helping the end effector complete its task more efficiently.

[0084] It should be noted that the above-mentioned processing method based on the three-dimensional spatial registration algorithm, distributed timestamp and least squares algorithm is only for illustrative purposes. It should be emphasized that this application is not limited to the specific technical methods adopted. Regardless of the technical means (such as Kalman filtering to calibrate time deviation, deep learning-based spatiotemporal feature alignment algorithm, etc.), as long as it can achieve time calibration and alignment of environmental information, eliminate the differences in spatial coordinate systems and time dimensions between different acquisition channels, and ensure the consistency, synchronization and accuracy of environmental information in the spatiotemporal dimensions, it can be used to construct the world model.

[0085] In one embodiment, extracting features from the calibration information to obtain the target feature information includes:

[0086] In the case where the calibration information is geometric point cloud data, performing feature extraction on the geometric point cloud data by a principal component analysis method to obtain eigenvalues ​​corresponding to the geometric point cloud data;

[0087] When the calibration information is a motion pose sequence of the end effector, extracting a joint friction coefficient of the motion pose sequence;

[0088] In the case where the calibration information is temperature field data, extracting core time-frequency information of temperature fluctuations in the temperature field data by Fourier transform;

[0089] The target feature information includes the feature value, the joint friction coefficient and the core time frequency information.

[0090] Optionally, in embodiments of this application, geometric point cloud data is a collection of discrete points used to represent a three-dimensional object or scene. Collected by devices such as lidar and depth cameras, each point contains three-dimensional spatial coordinates (X, Y, Z) and may also include information such as color and reflection intensity. These points can approximate the surface shape of an object and, in this application, are used to represent the geometry of an interactive environment, interactive object, or the end effector body.

[0091] The eigenvalue measures the degree of discreteness of the data in the direction of the corresponding principal component. The larger the value, the more original data information is contained in that direction, and it can reflect the key geometric characteristics of the point cloud data.

[0092] A motion pose sequence records the position and attitude of the end effector at a series of moments during its motion. Position refers to coordinates in three-dimensional space, while attitude can be expressed using rotation matrices, Euler angles, or quaternions, describing the orientation of the end effector. A motion pose sequence fully captures the end effector's trajectory and attitude changes.

[0093] When an end effector joint moves, friction is generated due to contact and relative motion between joint components. The joint friction coefficient is a proportional coefficient that measures the magnitude of this friction. It affects the smoothness of joint movement, energy consumption, and motion control accuracy, and is a key parameter reflecting the mechanical characteristics of the end effector.

[0094] Temperature field data is a collection of data describing the temperature distribution in space. It consists of temperature values ​​at different locations within a given spatial range, as well as how the temperature changes over time. This data can be collected through a network of temperature sensors and used to characterize the temperature state of the interaction environment, the end effector, or the interaction object.

[0095] For temperature field data, Fourier transforms are used to convert temperature fluctuation signals from the time domain into the frequency domain for analysis. Temperature fluctuations often contain multiple frequency components. Core time-frequency information refers to the frequency components with the largest energy share and the primary influence on temperature variation. It reflects the primary periodic characteristics and variation patterns of temperature fluctuations.

[0096] It should be noted that in the above embodiments, the principal component analysis method is used for geometric point cloud data, the joint friction coefficient is extracted from the motion posture sequence, and the temperature field data is processed by Fourier transform, etc., which are only examples of feature extraction. It should be clear that this application is not limited to the above-mentioned specific data types and extraction methods. Whether it is the pressure distribution and light intensity in the environment, or the vibration frequency and current consumption of the end effector, as long as the key features related to the interactive environment and the end effector state can be extracted from the calibration information through reasonable algorithms and technical means, they are applicable to this application.

[0097] Optionally, in a specific embodiment of the present application, as Figure 2 As shown, the end effector of this application includes three modules: a multimodal perception module, a multi-process additive manufacturing module, and an embodied intelligent body module based on deep learning.

[0098] Optionally, in a specific implementation, the first step is to obtain environmental information. In actual operation scenarios, various sensors in the multimodal perception module are used to collect information. A depth camera or laser scanner is used to collect geometric information such as the shape, size, color, texture, etc. of the interactive environment and interactive objects of the interactive task, and present it in the form of point cloud data, such as , which contains the three-dimensional coordinates and color information of each point. The temperature sensor is used to obtain the temperature field distribution of the interactive environment and interactive objects. Describes different locations in space The temperature condition at time t. The data exchange system of the multi-process additive manufacturing module drives the actuator body of the end effector to move, and collects its dynamic characteristics in the field environment, such as obtaining time series pose data It can also drive the main body to touch the object and use the tactile sensor to collect the mechanical properties of the object, such as the stress-strain relationship. ,In this way, the environmental information covering the geometric and physical information of the interactive environment, interactive objects, and the actuator body of the end effector is obtained.

[0099] Then, time calibration and feature extraction are performed. For the collected multi-source heterogeneous environmental information, a unified coordinate system is first established based on the three-dimensional spatial registration algorithm to obtain the information obtained from different collection channels, and the data under different spatial references are integrated into the same framework to obtain the first information. Then, the first information is time-calibrated using the distributed timestamp combined with the least squares algorithm to eliminate the time deviation caused by factors such as different sensor sampling times, and obtain calibration information. After that, feature extraction is performed on the calibration information. If the calibration information is geometric point cloud data, the principal component analysis method is used to find the direction with the largest data variance and obtain the corresponding eigenvalue, such as , these eigenvalues ​​reflect the key geometric features of the point cloud data; when the calibration information is the motion pose sequence of the end effector, the joint friction coefficient is extracted , which is used to reflect the friction characteristics of joint movement; if the calibration information is temperature field data, the temperature fluctuation signal in the time domain is converted to the frequency domain through Fourier transform to extract the core time-frequency information of the temperature fluctuation , in order to reflect the main periodic characteristics of temperature change. After these operations, the target feature information is obtained.

[0100] In these optional embodiments, principal component analysis processes geometric point cloud data and extracts eigenvalues, which can effectively compress data dimensions, highlight key geometric features, and facilitate the distinction of object shape structures. Extracting the joint friction coefficient of the motion posture sequence can accurately reflect the joint motion characteristics of the end effector and help optimize motion control and energy management. Fourier transform extracts core time and frequency information from temperature field data, reveals the law of temperature fluctuation, and provides a basis for decision-making related to ambient temperature. In summary, the target feature information obtained by these steps lays a solid foundation for building an accurate world model and improving the task execution capability of the end effector.

[0101] In one embodiment, constructing the world model according to the target feature information includes:

[0102] Reconstructing a three-dimensional geometric model of the interactive environment through implicit neural representation according to target feature information corresponding to the interactive environment;

[0103] According to the target feature information corresponding to the interactive object, a multi-physical property model of the interactive object is constructed through a material constitutive equation;

[0104] Determining a body dynamics model of the actuator body through a Lie group kinematic chain according to target feature information corresponding to the actuator body;

[0105] Determine the interaction dynamics model of the actuator body and the interaction environment through the Hamiltonian equation according to the body dynamics model;

[0106] constructing, based on the target feature information, an energy potential contact model describing the contact between the actuator body and the interactive object;

[0107] Constructing a state transition equation for predicting the multi-physics field evolution of the interactive environment based on the target feature information;

[0108] The world model is constructed according to the three-dimensional geometric model, the multi-physics attribute model, the body dynamics model, the interaction dynamics model, the energy potential contact model and the state transfer equation.

[0109] Optionally, in an embodiment of the present application, implicit neural representation is a method of representing geometric shapes or other continuous signals using a neural network. It can learn the underlying structure in the data and map the input target feature information to a continuous function. When constructing a three-dimensional geometric model of an interactive environment, the implicit neural representation can utilize the powerful fitting ability of the neural network to learn the geometric shape, surface features, etc. of the objects in the environment based on the target feature information corresponding to the environment, thereby reconstructing the three-dimensional geometric model.

[0110] A 3D geometric model is a mathematical description of the three-dimensional spatial structure of an interactive environment. It represents the shape, size, and positional relationships of objects within the environment using geometric elements such as points, lines, and surfaces. In this embodiment, an implicit neural representation method is used to construct a model based on target feature information corresponding to the interactive environment, providing a spatial foundation for subsequent analysis of the end effector's motion and interaction within the environment.

[0111] The body dynamics model describes the relationship between the motion and mechanical properties of the end effector's body. By using a Lie group kinematic chain and combining it with target characteristic information corresponding to the end effector's body (such as joint friction coefficient), the motion state, force conditions, and interactions of each joint can be determined. This allows the body dynamics model to be established for predicting and controlling the end effector's motion.

[0112] The interactive dynamics model studies the dynamic relationship between the end-effector's body and its interactive environment. Based on the body dynamics model, the Hamiltonian equations are used to consider factors such as force transmission and energy exchange between the end-effector and the environment. The model determines the dynamic behavior of the end-effector and the environment during the interaction process, helping to understand and optimize the end-effector's operation within the environment.

[0113] The energy potential contact model describes the energy and potential changes during contact between the end-effector and the interacting object. This model, constructed based on target feature information and taking into account factors such as force, deformation, and energy conversion during contact, analyzes the mechanical properties and energy distribution during contact, providing a basis for designing appropriate contact strategies and evaluating contact effectiveness.

[0114] Multiphysics evolution refers to the temporal evolution of multiple physical fields (such as temperature, electromagnetic, and stress) in an interactive environment. These fields influence and couple with each other, collectively determining the dynamic characteristics of the environment.

[0115] State transition equations are mathematical equations used to predict the evolution of multi-physics fields in an interacting environment. These equations can be used to predict the dynamic changes in the environment, enabling the end effector to make appropriate decisions and adjustments in advance.

[0116] Optionally, in one specific implementation, first, to construct a three-dimensional geometric model of the interactive environment, an implicit neural representation method can be used based on the target feature information corresponding to the interactive environment. This feature information is used as the input of the neural network. The powerful nonlinear fitting ability of the neural network is utilized to continuously train the network so that it learns the geometric shape, spatial layout, and other features of the objects in the interactive environment. The continuous function output by the neural network can describe any point in the environment, thereby reconstructing the three-dimensional geometric model of the interactive environment and accurately presenting the spatial structure of the environment.

[0117] Next, we construct a multi-physics property model for the interacting object. Based on the target feature information corresponding to the interacting object, we combine it with the material constitutive equation. This material constitutive equation describes the relationship between physical quantities such as stress, strain, and temperature. Material-related parameters (such as material type and physical properties) from the target feature information are substituted into the corresponding constitutive equation. Through calculation and analysis, we determine the multi-physics properties of the interacting object, such as mechanical and thermal properties, under different conditions. This allows us to construct a multi-physics property model that accurately reflects the physical characteristics of the interacting object.

[0118] Then, the dynamic model of the end effector's body is determined. Using the target feature information corresponding to the end effector's body, a Lie group kinematic chain is employed. A Lie group kinematic chain is a mathematical tool used to describe robot kinematics. By modeling the motion of each joint of the end effector and combining parameters such as the joint friction coefficient from the target feature information, it can accurately determine the motion state and force conditions of the end effector's body, thereby establishing a body dynamics model and achieving a precise description of the end effector's body motion.

[0119] Next, the interaction dynamics model is constructed. Based on the established body dynamics model, the Hamiltonian equation is applied. The Hamiltonian equation is a key equation in analytical mechanics, describing the dynamic behavior of a system from an energy perspective. The relevant parameters from the body dynamics model are substituted into the Hamiltonian equation. Factors such as energy exchange and force transfer between the end effector and the interaction environment are considered to calculate the dynamic relationship between the two during the interaction process, thereby determining the interaction dynamics model.

[0120] Next, an energy potential contact model is constructed. Based on the target's characteristic information, factors such as force, deformation, and energy conversion during contact between the end effector and the interacting object are considered. By establishing a mathematical model to describe the changes in energy and potential energy during contact and simulating contact kinematics, an energy potential contact model is constructed to analyze the mechanical properties and energy distribution during contact.

[0121] Next, we construct state transition equations. Based on the target's characteristic information, we consider the changing patterns of multiple physical fields (such as temperature and stress) within the interactive environment. By analyzing the historical data and current states of these physical fields, we establish equations that describe the relationship between the current and future states. This allows us to predict the evolution of these multiple physical fields within the interactive environment.

[0122] Finally, the 3D geometric model, multi-physics attribute model, body dynamics model, interaction dynamics model, energy potential contact model, and state transition equations constructed above are integrated. These models and equations describe the characteristics and relationships of the interactive environment, interactive objects, and the end-effector body from different perspectives. By integrating this information, a complete world model is constructed, providing comprehensive and accurate foundational support for the end-effector to perform the target interactive task.

[0123] Optionally, in the above embodiments, operations such as reconstructing a three-dimensional geometric model through implicit neural representation and constructing a multi-physical property model using material constitutive equations are only exemplary technical means for constructing a world model. It should be emphasized that the present application is not limited to the above-mentioned specific feature extraction dimensions and modeling methods. Whether it is to generate an environmental model using a topology optimization algorithm or to predict the behavior of interactive objects with the help of machine learning, as long as it can effectively characterize the physical characteristics of the interactive environment, the property changes of the interactive objects, the dynamic behavior of the end effector, and the dynamic interaction relationship between the three based on the target feature information, it can be used to construct a world model.

[0124] In these optional embodiments, implicit neural representations reconstruct 3D geometric models, accurately reproducing the spatial structure of the interactive environment. Material constitutive equations construct multi-physics property models, clarifying the physical properties of interacting objects. Lie group kinematic chains and Hamiltonian equations define dynamic models, understanding the motion and interaction between the actuator and the environment. Energy potential contact models analyze contact characteristics, and state transition equations predict environmental changes. The world model constructed by integrating these models provides a reliable basis for decision-making and control of the end effector in complex tasks, improving the accuracy and efficiency of task execution.

[0125] In one embodiment, the first preset model is trained to maximize the execution success rate of the target interaction task, including:

[0126] constructing a first initial model, the first initial model taking into account environmental constraints of the interactive environment and diversity of task requirements of the interactive task;

[0127] The first initial model is trained using a first preset training sample with the goal of maximizing the task execution success rate until a first preset training stop condition is satisfied, thereby obtaining the first preset model; the first preset model is used to determine a target structure and an interactive behavior strategy, wherein the structural characteristics of the target structure include the kinematic accessibility, mechanical stiffness, and compliance of the target structure; and the interactive behavior strategy includes a basic action sequence and a safety strategy library;

[0128] The second preset model is trained to maximize the structural quality of the target structure and the interaction robustness of performing the target interaction task, and includes:

[0129] Constructing a second initial model;

[0130] The second initial model is trained using second preset training samples with the goal of maximizing the structural quality of the target structure and the interactive robustness of performing the target interactive task until a second preset training stop condition is met, thereby obtaining the second preset model; wherein, when the second preset model determines the structural manufacturing instructions and the quality control rules, path continuity and obstacle avoidance are considered; and when the second preset model determines the interactive motion control instructions and the feedback compensation mechanism, different failure modes of the end effector are considered.

[0131] Optionally, in the embodiments of the present application, environmental constraints of the interactive environment refer to the constraints imposed by the interactive environment on the end effector and the execution of the task during the execution of the interactive task. For example, when working in a confined space, the size of the space may limit the range of motion of the end effector; in environments with high temperature, high humidity, or strong electromagnetic interference, special environmental factors may affect the performance of the end effector's electronic components and material properties, thereby hindering the execution of the task.

[0132] Diversity in task demands refers to the diverse requirements of different interactive tasks. For example, in industrial production, assembly tasks require extremely high precision, requiring the end effector to precisely manipulate parts; while handling tasks prioritize load capacity and motion efficiency; and in home service scenarios, cleaning tasks require the end effector to adapt to the surface materials and shapes of different furniture. These differentiated requirements reflect the diversity of task demands.

[0133] Kinematic reachability describes the ability of an end-effector to achieve the desired position and posture in space. A target structure with good kinematic reachability means that the end-effector can freely move to the desired position in the task space and adjust to the appropriate posture to complete various actions such as grasping and placing objects without being restricted by its own structure.

[0134] Mechanism stiffness reflects the end effector's ability to resist deformation. When required to withstand large external forces or perform high-precision operations, sufficient mechanism stiffness ensures structural stability, maintains accurate motion trajectory and operational precision, and avoids excessive deformation due to force that could affect task execution.

[0135] Mechanism compliance, the opposite of stiffness, refers to the ability of an end-effector structure to deform in response to external forces. When interacting with delicate objects or adapting to complex surfaces, compliance prevents damage while providing a better fit and stable operation.

[0136] A basic action sequence is an ordered combination of basic actions performed by the end effector to complete a target interaction task. For example, in a grasping task, a basic action sequence might include steps such as approaching the object, adjusting the grasping posture, executing the grasping action, and lifting the object. These actions are performed sequentially in a certain order, forming the basic process for completing the task.

[0137] The safety policy library contains a series of rules and policies designed to ensure the safe operation of the end effector during mission execution. For example, force thresholds can be set so that when the detected contact force exceeds the threshold, the end effector stops moving to prevent damage to itself, the interacting object, or the surrounding environment. Alternatively, emergency response strategies can be developed to ensure system safety in the event of unexpected situations (such as power outages or sensor failures).

[0138] Path continuity requires that the end effector's motion path be continuous and smooth, avoiding sudden jumps or interruptions. This ensures motion stability, reduces mechanical shock, improves motion efficiency, and facilitates precise motion control.

[0139] Obstacle avoidance means that the end effector can detect obstacles in the surrounding environment in real time during movement, and plan a reasonable path to avoid them, prevent collision accidents, and ensure the safe and smooth execution of the task.

[0140] Different failure modes refer to various potential failure scenarios that may occur during the operation of the end effector. For example, motor failure may prevent joint motion from being properly driven; sensor failure may lead to inaccurate environmental information; and communication failure may prevent control commands from being properly transmitted. Considering these failure modes when determining interactive motion control commands and feedback compensation mechanisms allows the end effector to take appropriate measures to maintain operation or safely shut down when a failure occurs, thereby improving system reliability and stability.

[0141] Optionally, in a specific implementation of the present application, a first initial model is first constructed, which fully considers the environmental constraints of the interactive environment and the diversity of task requirements of the interactive task. When constructing the model, a deep learning method is used to build a framework with a specific network structure to lay the foundation for subsequent learning. Then, the first initial model is trained using the first preset training sample, with the goal of maximizing the execution success rate of the target interactive task. The training process is based on the objective function:

[0142] ;

[0143] is the optimization direction, where 、 is the weight coefficient, Measure the actual position of the end effector With target location The deviation, Represents the magnitude of the contact force. By continuously adjusting the model parameters, the model learns how to generate the optimal target structure and interaction behavior strategy based on different environments and task conditions.

[0144] In terms of the target structure, kinematic accessibility, mechanical stiffness, and compliance must be comprehensively considered. When the first preset training stop condition is met, such as when the model's loss function converges to a certain level or the accuracy no longer improves, the first preset model is obtained. This first preset model aims to maximize the task execution success rate and conducts a quantitative evaluation of the task execution success rate:

[0145] ;

[0146] in, is the task execution success rate, The interactive behavior strategy is generated based on the target structure. The final result is the target structure under the optimized design, covering the geometric configuration, material distribution, kinematic constraints, etc., as well as the interactive behavior strategy, such as the target pose sequence, expected interactive contact force threshold, and safety rule library.

[0147] A second initial model is constructed as the basic framework. It is trained using the second preset training samples, with the goal of maximizing the structural quality of the target structure and the interactive robustness of performing the target interactive task. During the training process, the model learns to generate structural manufacturing instructions and quality control rules, taking into account path continuity and obstacle avoidance. When determining the interactive motion control instructions and feedback compensation mechanism, the model considers different failure modes of the end effector, such as but not limited to collision, slippage, failure, etc. With the goal of maximizing part quality and interactive robustness, factors such as process constraints, environmental interference, and estimation errors are considered:

[0148] ;

[0149] in, represents the structural quality of the target structure, represents the interaction robustness, represent process constraints, environmental interference and estimation errors respectively.

[0150] When the second preset training stop condition is met, a second preset model is generated. This second preset model generates additive manufacturing, interactive action instructions, and closed-loop feedback control logic, including system motion control instructions and real-time compensation mechanisms, ultimately achieving the desired actuator structure and completing the interactive task. Through the training and optimization of these two models, from structural design and behavior planning to manufacturing and interactive control, the end effector is fully guaranteed to complete its tasks efficiently and safely in complex environments.

[0151] In these optional embodiments, the first preset model takes into account environmental constraints and the diversity of task requirements, and determines the target structure and interactive behavior strategy through training, which can improve the success rate of executing the target interactive task. A reasonable target structure ensures kinematic accessibility, mechanical stiffness and compliance, and a suitable interactive behavior strategy provides a basic action sequence and a safety strategy library, so that the actuator can better adapt to complex tasks. The second preset model focuses on structural quality and interactive robustness, and considers paths, obstacles and failure modes when determining instructions and rules, which can ensure the manufacture of high-quality structures, enhance stability and reliability when executing tasks, reduce the impact of failures, and improve the performance and practicality of the end effector as a whole.

[0152] In one embodiment, controlling the end effector to manufacture the target structure on demand in real time through a multi-process additive manufacturing function based on the world model, the target structure, the structure manufacturing instruction, and the quality control rule includes:

[0153] During the process of the end effector manufacturing the target structure according to the structure manufacturing instruction, based on the world model, according to the manufacturing state of the end effector at a current moment and the quality control rule, the structure manufacturing instruction is adjusted so that the manufacturing quality of the target structure meets a first preset quality.

[0154] In one embodiment, controlling the target actuator to perform the target interaction task according to the interactive behavior strategy, the interactive motion control instruction, and the feedback compensation mechanism includes:

[0155] In the process of the target actuator performing the target interactive task according to the interactive motion control instruction, based on the world model, the interactive motion control instruction is adjusted according to the interactive state of the target actuator at the current moment and the feedback compensation mechanism, so that the execution quality of the target interactive task performed by the end actuator meets the second preset quality.

[0156] Alternatively, as Figure 2 As shown, in terms of target structure manufacturing, based on the generated structural manufacturing instructions, the motion control system and material supply system of the multi-process additive manufacturing module work together to start executing pre-planned manufacturing instructions to create the target structure. During the manufacturing process, the data exchange system feeds back the system status to the world model in real time. The world model evaluates and analyzes the manufacturing quality based on the manufacturing status of the end effector at the current moment, such as the material deposition position, structure forming progress, etc., as well as the quality control rules. If factors that may affect the quality are found in the manufacturing process, such as uneven material deposition, structural size deviation, etc., the structural manufacturing instructions will be adjusted based on this information. For example, the path planning of the motion control system is adjusted to make the material deposition more uniform, or the material supply speed is changed to ensure that the manufacturing quality of the target structure meets the first preset quality, thereby achieving efficient and high-quality target structure manufacturing.

[0157] During the interactive task execution phase, the manufactured target actuator is driven to begin motion according to the interactive motion control instructions. The multimodal perception module collects system information in real time, including the target actuator's position, posture, and contact status with the interactive object, and transmits this information to the world model. The world model evaluates the interactive motion control instructions based on the target actuator's current interactive state using a feedback compensation mechanism. If anomalies are detected during the interaction process, such as excessive contact force or slippage, the world model adjusts the interactive behavior in real time based on preset feedback compensation criteria. This may involve adjusting contact impedance parameters to change the compliance of the target actuator in contact with the interactive object, or performing reverse displacement compensation to correct motion deviations of the end effector, ensuring high-quality execution of the interactive task. Simultaneously, a task evaluation report is generated to provide data support for the continued intelligent growth of embodied intelligence and continuously optimize the performance of the target actuator in subsequent tasks.

[0158] In these optional embodiments, when manufacturing the target structure, structural manufacturing instructions are adjusted based on the world model, combined with manufacturing status and quality control rules. This allows for timely correction of manufacturing deviations, ensuring that the target structure's manufacturing quality meets the first preset quality, and preventing subsequent tasks from being impacted by manufacturing defects. When executing interactive tasks, interactive motion control instructions are adjusted based on the world model, interaction status, and feedback compensation mechanisms. This enables the target actuator to quickly adapt to task changes, effectively respond to emergencies, and ensure that execution quality meets the second preset quality. Overall, these steps improve the adaptability and reliability of the target actuator, ensuring accurate and efficient task completion.

[0159] In a specific embodiment of the present application, an end effector is provided, the technical solution of which mainly includes a multimodal perception module, a multi-process additive manufacturing module, and an embodied intelligent agent based on deep learning. The multimodal perception module is used to perceive the real-time scene conditions to collect physical information of the environment and the operating objects, and to construct a virtual world model of the task scene. It is the decision-making basis for the actuator morphology design and behavior reasoning; the multi-process additive manufacturing module is responsible for the on-demand on-site manufacturing of the end effector, and is the functional basis for realizing the self-growth of the end effector; the embodied intelligent agent based on deep learning is responsible for generating the required structure according to the task and environmental requirements, reasoning about the object interaction behavior, and controlling the manufacturing-interaction process with the goal of maximizing the success rate of task execution. The present invention realizes the on-demand autonomy of the end effector in various unstructured environments, replacing the traditional task / environment-oriented design of the actuator. It can effectively solve the challenge of requiring a variety of end effectors under existing unstructured environments and limited resource conditions, expand the feasible environment range of robot compliant manipulation, and improve the success rate of task execution.

[0160] The multimodal perception module collects on-site data to build the virtual world model required for the task, which includes information related to the geometric and physical properties of the environment, manipulated objects, and the operation entity.

[0161] The multi-process additive manufacturing module forms the required structure by directly depositing materials layer by layer on the actuator base, or continues to deposit materials on the existing structure to form the required structure, or continues to deposit materials after partially removing the existing structure to form the required structure. The materials involved include but are not limited to thermoplastics, thermosetting plastics, fiber-reinforced composites, metal / ceramic slurry, liquid metal, metal powder, etc., and the manufacturing processes used include but are not limited to fused deposition modeling, direct ink writing, directed energy deposition, etc.

[0162] like Figure 2 As shown in the figure, it mainly includes three modules: multimodal perception module (steps 1-3), multi-process additive manufacturing module (steps 4-5), and embodied intelligent agent based on deep learning (steps 6-7). The specific implementation is as follows:

[0163] Step 1: On-site data collection. At the operation site, a variety of sensors equipped with a multimodal perception system are used to collect geometric and physical information of the operation environment, operation objects, and operation entities. Specifically, it includes but is not limited to using depth cameras or laser scanners to collect information such as the shape, size, color, and texture of the environment and objects, such as ; Use temperature sensors to collect the temperature field distribution of the environment and objects, such as ; Drive the motion of the body through the data exchange system of the multi-process additive manufacturing module to collect the dynamic characteristics of the body in the field environment, such as time series pose data ; By driving the main body to touch the operation object and combining the tactile sensor to collect the mechanical properties of the operation object, such as stress-strain relationship wait.

[0164] Step 2: Multimodal data analysis. For the multi-source heterogeneous data collected in the previous step, the sensor data is time-calibrated using a distributed timestamp combined with a least squares algorithm, and a unified coordinate system is established based on a three-dimensional spatial registration algorithm to eliminate the temporal and spatial offset errors between different sensors. The corresponding features are extracted from the data after temporal and spatial alignment, including but not limited to extracting feature values ​​using principal component analysis for geometric point cloud data. ; Extract joint friction coefficient based on the body motion posture sequence etc.; extract the core time-frequency information of temperature fluctuations through Fourier transform of temperature field data ;

[0165] Step 3: World Model Construction. Based on the collected data and extracted features, an interpretable and predictable multi-physics digital twin of the operational scenario is established. This includes, but is not limited to, reconstructing the 3D geometry of the environment through implicit neural representation and modeling the multi-physics properties of the object in combination with material constitutive equations; describing the kinematics of the object using Lie group motion chains and embedding Hamiltonian equations to characterize the dynamics of the environmental interaction; simulating contact kinematics based on an energy potential contact model, and constructing state transition equations to predict the evolution of multi-physics fields.

[0166] Step 4: Structural design and behavioral reasoning. Based on the task requirements and environmental constraints, an autonomous policy network built based on deep learning methods is used to determine the objective function:

[0167] ;

[0168] Optimize the end-effector structure, including but not limited to kinematic accessibility, mechanism stiffness and compliance, and generate interactive behavior strategies , including but not limited to basic action sequences, security policy libraries, etc.

[0169] Step 5: Generate a manufacturing-interaction process control strategy. Based on the actuator structure generated in the previous step, consider factors such as path continuity and obstacle avoidance to generate structural manufacturing instructions and online quality control criteria. Based on the interactive behavior strategy generated in the previous step, consider failure modes such as collision, slippage, and failure to generate interactive motion control instructions and feedback compensation criteria.

[0170] Step 6: Multi-process additive manufacturing. Based on the generated structural manufacturing instructions, the motion control system and material supply system of the multi-process additive manufacturing module are driven to execute the pre-planned manufacturing instructions to create the target structure. The system status is fed back to the constructed world model in real time through the data exchange system for manufacturing quality assessment and analysis. Combined with the established online quality control criteria, the target structure can be manufactured efficiently and with high quality.

[0171] Step 7: Intelligent Interaction Behavior Control. Based on the generated interactive action instructions, the end effector of the online manufacturing process is driven. The system information collected in real time by the multimodal perception module is used to evaluate the current state and predict the next stage state through the world model. The interactive behavior is adjusted in real time according to the established feedback compensation criteria, including but not limited to contact impedance parameter adjustment and reverse displacement compensation. This ensures high-quality execution of the interactive task and generates a task evaluation report for the continuous intelligent growth of embodied intelligence.

[0172] like Figure 3 As shown, case 1 of the end effector structure in the simulation environment is shown.

[0173] Motion joint part:

[0174] The first Z-axis motion joint 1 and the second Z-axis motion joint 7 can move the end effector up and down in the Z-axis direction and adjust the vertical height position to adapt to different operational requirements, such as controlling the distance between the first print head 10 and the first additive manufacturing substrate 9 and the second additive manufacturing substrate 11 during additive manufacturing, or adjusting the height when grasping an object.

[0175] X-axis motion joint 3: responsible for the horizontal movement of the end effector in the X-axis direction, used to adjust the position left and right on the horizontal plane and expand the lateral range of operation.

[0176] The first and second Y-axis joints 8 and 12 enable horizontal movement of the end effector along the Y-axis, adjusting its position forward and backward. Together with the X-axis joint 3, they enable precise positioning of the end effector on a plane. These three joints work together to allow the end effector to flexibly move in three-dimensional space to its target location.

[0177] Perception components:

[0178] First depth camera 2: It can obtain depth information of the surrounding environment and objects, construct a three-dimensional scene, and is used to identify the shape, position, and distance of objects, etc., and provide visual information for the operation of the end effector, such as assisting in judging the position and posture of objects in grasping tasks.

[0179] The first temperature sensor 6 is used to monitor the temperature conditions during the additive manufacturing process, such as the temperature of the first print head 10 and the temperature field history of the manufacturing structure, to ensure that the manufacturing process is carried out within a suitable temperature range, to ensure the quality of material molding, and to avoid manufacturing defects caused by abnormal temperature.

[0180] Manufacturing related parts:

[0181] The first material supply device 5 is responsible for providing the required materials for additive manufacturing and delivering the materials to the first print head 10. It is the material supply source for additive manufacturing.

[0182] The first print head 10 deposits the material layer by layer according to the instructions to realize the additive manufacturing process of the self-growing end effector and construct the target structure.

[0183] The first additive manufacturing substrate 9 and the second additive manufacturing substrate 11 serve as basic platforms for additive manufacturing, provide a support surface for material deposition, carry the printed objects, and ensure the stability of the manufacturing process.

[0184] Basic support components:

[0185] The first actuator base 4 is the basic supporting structure of the entire end effector, providing a mounting base and support for other components, ensuring the relative position stability of each component and maintaining the stability of the overall structure.

[0186] like Figure 4 Figure 2 shows the end effector structure in a simulation environment. Compared to Example 1, the motion and control components are replaced with first servo 22, second servo 24, and third servo 210. These servo components act as power drive elements, precisely controlling the rotation angle and speed of first and second rotary joints 23 and 25. By receiving control signals, they provide power support for the end effector's motion, enabling precise posture adjustment and position shifting, and are key components for flexible operation.

[0187] The first rotary joint 23 and the second rotary joint 25 cooperate with the first servo 22, the second servo 24 and the third servo 210 to enable the corresponding components of the end effector to rotate around the axis, thereby increasing the freedom of movement of the end effector in space and adjusting the directions of components such as the second print head 212 and the second depth camera 26 to adapt to different operational requirements.

[0188] The second depth camera 26, the second temperature sensor 29, the second print head 212, the third additive manufacturing substrate 21 and the fourth additive manufacturing substrate 211, the second material supply device 28, and the second actuator base 27 can be explained with reference to the above. Figure 3 The explanation is omitted here.

[0189] like Figure 5 Figure 3 shows the end effector structure in a simulation environment. Mobile chassis 31: Provides mobility for the entire system, allowing for flexible movement within the work environment. This allows the end effector to reach different working positions, expanding the operating range and making it suitable for scenarios where tasks need to be performed within a larger space.

[0190] Robot 32: Equipped with multiple joints and degrees of freedom, it can perform complex motion operations. It can precisely control the position and posture of the end effector to accurately execute pre-defined actions such as grasping, placing, and manipulating tools in additive manufacturing or other interactive tasks.

[0191] Additive Manufacturing Module 33: This is the core component for implementing additive manufacturing of self-growing end-effectors. By stacking materials layer by layer and building the target structure according to the design model, end-effector components can be manufactured to meet different mission requirements, providing the hardware foundation for the end-effector's functional realization.

[0192] Multi-process tool library 34: Stores tools for various processes, allowing for rapid tool change based on task requirements. For example, in additive manufacturing, different print heads or processing tools can be switched as needed, while other tasks can also be replaced with appropriate operating tools, enhancing the end effector's versatility and adaptability.

[0193] During the manufacturing process, the end effector 35, gradually formed by the additive manufacturing module, is a key component that ultimately performs specific tasks. Its structure and functions are customized according to the task requirements. After manufacturing, combined with the robot's motion control, it can perform various interactive tasks such as assembly, handling, and inspection.

[0194] like Figure 6 As shown in Figure 1, a schematic diagram of three interaction strategies for the sphere in case 1 of the end effector structure is shown. Figure 6 (a) is a schematic diagram of sphere grasping; Figure 6 (b) is a schematic diagram of sphere adsorption; Figure 6 (c) in the figure is a schematic diagram of lifting the sphere.

[0195] like Figure 7 As shown, two installation and application methods of the end effector structure case 1 are shown:

[0196] Figure 7 (a) shows a collaborative robot arm installation application. The self-growing end effector is installed on the collaborative robot arm, which has multiple flexible joints.

[0197] Figure 7 (b) shows a drone installation application. The self-growing end effector is mounted on the bottom of the drone, which uses multiple rotors to provide flight power, enabling it to flexibly fly and hover in the air.

[0198] like Figure 8 As shown, an interactive application of the end effector structure case 3 is a collaborative robot arm installation and sphere grasping application.

[0199] Figure 9 A structural schematic diagram of an end effector execution device provided in another embodiment of the present application is shown. For the sake of convenience, only the parts related to the embodiment of the present application are shown.

[0200] Reference Figure 9 , the end effector execution device may include:

[0201] A construction module 901 is configured to construct a world model based on the environmental information of the end effector performing the target interactive task; the world model is configured to characterize the physical and dynamic characteristics of the interactive environment in which the end effector performs the target interactive task;

[0202] A first determination module 902 is configured to determine, based on a first preset model, a target structure of the end effector and an interactive behavior strategy for the end effector to perform the target interactive task according to the world model and the task requirements of the end effector; the first preset model is trained to maximize the execution success rate of the target interactive task;

[0203] A second determination module 903 is configured to determine, based on a second preset model and the target structure, structural manufacturing instructions for the end effector to manufacture the target structure, as well as quality control rules for the end effector to manufacture the target structure; and, based on the interactive behavior strategy, determine interactive motion control instructions for the end effector to execute the interactive behavior strategy, as well as a feedback compensation mechanism for the end effector to execute the target interactive task according to the interactive motion control instructions; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task;

[0204] A first control module 904 is configured to control the end effector to manufacture the target structure in real time on demand using a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instructions, and the quality control rules;

[0205] The second control module 905 is used to control the target actuator to perform the target interaction task according to the interactive behavior strategy, the interactive motion control instructions and the feedback compensation mechanism when the target structure is manufactured. The target actuator is the end actuator having the target structure.

[0206] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the above-mentioned method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device. Its specific functions and the technical effects brought about can be found in the method embodiment part, which will not be repeated here.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0208] Figure 10A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0209] The device may include a processor 1001 and a memory 1002 storing program instructions.

[0210] When the processor 1001 executes the program, the steps in any of the above method embodiments are implemented.

[0211] For example, the program can be divided into one or more modules / units, one or more modules / units are stored in the memory 1002 and executed by the processor 1001 to complete the present application. One or more modules / units can be a series of program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the program in the device.

[0212] Specifically, the processor 1001 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0213] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 1002 may include removable or non-removable (or fixed) media. Where appropriate, the memory 1002 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1002 is a non-volatile solid-state memory.

[0214] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, generally, the memory includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0215] The processor 1001 implements any one of the methods in the above embodiments by reading and executing program instructions stored in the memory 1002 .

[0216] In one example, the electronic device may further include a communication interface 1003 and a bus 1010. The processor 1001, the memory 1002, and the communication interface 1003 are connected via the bus 1010 and communicate with each other.

[0217] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0218] Bus 1010 includes hardware, software or both, and the parts of online data flow metering equipment are coupled to each other. For example, but not limitation, bus can include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1010 can include one or more buses. Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0219] In addition, in combination with the methods in the above embodiments, embodiments of the present application may provide a storage medium for implementation. The storage medium stores program instructions; when the program instructions are executed by a processor, any one of the methods in the above embodiments is implemented.

[0220] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0221] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0222] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0223] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0224] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in unit, a function card or the like. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium that can store or transmit information. The example of a machine-readable medium includes an electronic circuit, a semiconductor memory device, a ROM, a flash memory, an erasable ROM (EROM), a floppy disk, a CD-ROM, an optical disk, a hard disk, an optical fiber medium, a radio frequency (RF) link, or the like. The code segment can be downloaded via a computer grid such as the Internet, an intranet, etc.

[0225] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0226] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed via the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. This processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or the flowchart and the combination of the boxes in the block diagram and / or the flowchart can also be implemented by the dedicated hardware that performs the specified function or action, or can be implemented by the combination of dedicated hardware and computer instructions.

[0227] The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application.

Claims

1. An end effector execution method, characterized in that: The method comprises: Constructing a world model based on the environmental information of the end effector performing the target interactive task; the world model is used to characterize the physical and dynamic characteristics of the interactive environment in which the end effector performs the target interactive task; Based on a first preset model, determining a target structure of the end effector and an interactive behavior strategy for the end effector to perform the target interactive task according to the world model and the task requirements of the end effector; the first preset model is trained with the goal of maximizing the execution success rate of the target interactive task; Based on the second preset model, according to the target structure, structural manufacturing instructions for the end effector to manufacture the target structure and quality control rules for the end effector to manufacture the target structure are determined; and according to the interactive behavior strategy, interactive motion control instructions for the end effector to execute the interactive behavior strategy and a feedback compensation mechanism for the end effector to execute the target interactive task according to the interactive motion control instructions are determined; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task; Controlling the end effector to manufacture the target structure on demand in real time through a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instructions, and the quality control rules; When the target structure is manufactured, the target actuator is controlled to perform the target interaction task according to the interactive behavior strategy, the interactive motion control instruction and the feedback compensation mechanism. The target actuator is an end actuator having the target structure.

2. The method according to claim 1, characterized in that The process of constructing a world model based on the environmental information of the end effector performing the target interaction task includes: Acquiring the environmental information, wherein the environmental information includes geometric information and physical information of the interactive environment, the interactive object of the interactive task, and the actuator body of the end effector; Performing time calibration and feature extraction on the environmental information to obtain target feature information; The world model is constructed according to the target feature information, and the world model is used to realize dynamic perception prediction of the interactive environment.

3. The method according to claim 2, characterized in that The performing time calibration and feature extraction on the environmental information to obtain target feature information includes: Establishing a unified coordinate system for the environmental information collected from different collection channels based on a three-dimensional space registration algorithm to obtain first information; Performing time calibration on the first information by using a distributed timestamp combined with a least squares algorithm to obtain calibration information; Feature extraction is performed on the calibration information to obtain the target feature information.

4. The method according to claim 3, characterized in that The extracting features from the calibration information to obtain the target feature information includes: In the case where the calibration information is geometric point cloud data, performing feature extraction on the geometric point cloud data by a principal component analysis method to obtain eigenvalues ​​corresponding to the geometric point cloud data; When the calibration information is a motion pose sequence of the end effector, extracting a joint friction coefficient of the motion pose sequence; In the case where the calibration information is temperature field data, extracting core time-frequency information of temperature fluctuations in the temperature field data by Fourier transform; The target feature information includes the feature value, the joint friction coefficient and the core time frequency information.

5. The method according to claim 2, characterized in that The constructing the world model according to the target feature information includes: Reconstructing a three-dimensional geometric model of the interactive environment through implicit neural representation according to target feature information corresponding to the interactive environment; According to the target feature information corresponding to the interactive object, a multi-physical property model of the interactive object is constructed through a material constitutive equation; Determining a body dynamics model of the actuator body through a Lie group kinematic chain according to target feature information corresponding to the actuator body; Determine the interaction dynamics model of the actuator body and the interaction environment through the Hamiltonian equation according to the body dynamics model; constructing, based on the target feature information, an energy potential contact model describing the contact between the actuator body and the interactive object; Constructing a state transition equation for predicting the multi-physics field evolution of the interactive environment based on the target feature information; The world model is constructed according to the three-dimensional geometric model, the multi-physics attribute model, the body dynamics model, the interaction dynamics model, the energy potential contact model and the state transfer equation.

6. The method according to claim 1, characterized in that The first preset model is trained to maximize the execution success rate of the target interaction task, and includes: constructing a first initial model, the first initial model taking into account environmental constraints of the interactive environment and diversity of task requirements of the interactive task; The first initial model is trained using a first preset training sample with the goal of maximizing the task execution success rate until a first preset training stop condition is satisfied, thereby obtaining the first preset model; the first preset model is used to determine a target structure and an interactive behavior strategy, wherein the structural characteristics of the target structure include the kinematic accessibility, mechanical stiffness, and compliance of the target structure; and the interactive behavior strategy includes a basic action sequence and a safety strategy library; The second preset model is trained to maximize the structural quality of the target structure and the interaction robustness of performing the target interaction task, and includes: Constructing a second initial model; The second initial model is trained using second preset training samples with the goal of maximizing the structural quality of the target structure and the interactive robustness of performing the target interactive task until a second preset training stop condition is met, thereby obtaining the second preset model; wherein, when the second preset model determines the structural manufacturing instructions and the quality control rules, path continuity and obstacle avoidance are considered; and when the second preset model determines the interactive motion control instructions and the feedback compensation mechanism, different failure modes of the end effector are considered.

7. The method according to claim 1, characterized in that The controlling the end effector to manufacture the target structure in real time on demand through a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instruction, and the quality control rule includes: During the process of the end effector manufacturing the target structure according to the structure manufacturing instruction, based on the world model, according to the manufacturing state of the end effector at a current moment and the quality control rule, the structure manufacturing instruction is adjusted so that the manufacturing quality of the target structure meets a first preset quality.

8. The method according to claim 1, characterized in that The controlling the target actuator to perform the target interaction task according to the interactive behavior strategy, the interactive motion control instruction, and the feedback compensation mechanism includes: In the process of the target actuator executing the target interactive task according to the interactive motion control instruction, based on the world model, the interactive motion control instruction is adjusted according to the interactive state of the target actuator at the current moment and the feedback compensation mechanism, so that the execution quality of the target actuator executing the target interactive task meets the second preset quality.

9. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the end effector execution method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the end effector execution method according to any one of claims 1 to 8 is implemented.

Images