End effector execution method and device and medium

Abstract

The invention provides an end effector execution method and device and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: building a world model according to the environment information of an end effector for executing a target interaction task; based on a first preset model, according to the world model and task requirements of the end effector, determining a target structure of the end effector and an interaction behavior strategy of the end effector for executing the target interaction task; based on a second preset model, according to the target structure, determining a structure manufacturing instruction of the end effector manufacturing target structure and a quality regulation and control rule of the end effector manufacturing target structure; controlling an end effector to manufacture a target structure in real time as required through a multi-process additive manufacturing function; and under the condition that the manufacturing of the target structure is completed, according to the interaction behavior strategy, the interaction motion control instruction and the feedback compensation mechanism, controlling a target actuator to execute a target interaction task. The task execution effect of the end effector can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an end effector execution method, device and medium. Background Art

[0002] As a key component for robots to directly perform tasks, the performance and adaptability of the end effector directly determine the robot's ability to complete tasks. In unstructured scenarios such as mass customized factories, home services, and field exploration, the end effector needs to cope with complex and changing environments and diverse operating tasks. At present, the end effector execution method for unstructured environments is difficult to adapt to complex environments and task requirements. In these unstructured scenarios, complex situations such as strong coupling of geometric and physical properties of the operating objects, rigid-flexible composite, diverse surface characteristics, and temperature and humidity constraints on the contact surface frequently occur, which puts strict requirements on the control accuracy and strategy robustness of robot manipulation. The existing technology mainly realizes object interaction by customizing soft robots and using air pressure or hydraulic pressure to drive the deformation of soft materials. However, the structure customized in this way is designed according to specific tasks and environments, lacks universality and adaptability, and is difficult to quickly adjust according to different geometric features and physical properties, resulting in the end effector. When performing tasks, the control accuracy is difficult to guarantee, the robustness of the interaction strategy is insufficient, and ultimately leads to poor task execution. Summary of the invention

[0003] The present application provides an end effector execution method, device and medium, which can improve the execution effect of the end effector in executing tasks.

[0004] In a first aspect, an embodiment of the present application provides an end effector execution method, the method comprising: A world model is constructed based on the environmental information of the end effector performing the target interactive task; the world model is used to characterize the physical and dynamic characteristics of the interactive environment in which the end effector performs the target interactive task; Based on the first preset model, according to the world model and the task requirements of the end effector, the target structure of the end effector and the interactive behavior strategy of the end effector to perform the target interactive task are determined; the first preset model is trained with the goal of maximizing the execution success rate of the target interactive task; Based on the second preset model, according to the target structure, the structural manufacturing instructions for the end effector to manufacture the target structure and the quality control rules for the end effector to manufacture the target structure are determined; and according to the interactive behavior strategy, the interactive motion control instructions for the end effector to execute the interactive behavior strategy and the feedback compensation mechanism for the end effector to execute the target interactive task according to the interactive motion control instructions are determined; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task; Control the end effector to manufacture the target structure in real time on demand through multi-process additive manufacturing functions according to the world model, the target structure, the structure manufacturing instructions, and the quality control rules; In the case where the target structure is manufactured, control the target actuator to execute the target interaction task according to the interaction behavior strategy, the interaction motion control instruction, and the feedback compensation mechanism, where the target actuator is the end effector with the target structure.

[0005] In a second aspect, the present application provides an end effector execution device, and the device includes: A construction module, configured to construct a world model according to the environmental information of the end effector executing the target interaction task; the world model is used to characterize the physical characteristics and dynamic characteristics of the interaction environment of the end effector executing the target interaction task; A first determination module, configured to determine the target structure of the end effector and the interaction behavior strategy of the end effector executing the target interaction task based on a first preset model according to the world model and the task requirements of the end effector; the first preset model is trained with the goal of maximizing the execution success rate of the target interaction task; A second determination module, configured to determine the structure manufacturing instructions for the end effector to manufacture the target structure and the quality control rules for the end effector to manufacture the target structure based on a second preset model according to the target structure; and determine the interaction motion control instruction for the end effector to execute the interaction behavior strategy and the feedback compensation mechanism for the end effector to execute the target interaction task according to the interaction motion control instruction according to the interaction behavior strategy; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interaction robustness of executing the target interaction task; A first control module, configured to control the end effector to manufacture the target structure in real time on demand through multi-process additive manufacturing functions according to the world model, the target structure, the structure manufacturing instructions, and the quality control rules; A second control module, configured to, in the case where the target structure is manufactured, control the target actuator to execute the target interaction task according to the interaction behavior strategy, the interaction motion control instruction, and the feedback compensation mechanism, where the target actuator is the end effector with the target structure.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, and the electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the end effector execution method in any one of the embodiments in the first aspect.

[0007] Fourthly, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the end effector execution method in any one of the embodiments in the first aspect is implemented.

[0008] Fifthly, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the end effector execution method in any one of the above-mentioned first aspects.

[0009] In an end effector execution method, device and medium provided by an embodiment of the present application, by constructing a world model to comprehensively represent the physical and dynamic characteristics of the interaction environment, the actual situation of the interaction environment can be accurately reflected. Based on a first preset model trained with the goal of maximizing the task execution success rate, the target structure and interaction behavior strategy are determined by combining the world model and task requirements. A second preset model trained with the goal of improving the structure quality and interaction robustness determines the structure manufacturing instructions, quality control rules, interaction motion control instructions and feedback compensation mechanism. Thus, when manufacturing the target structure, the quality control rules ensure the reliability of the target structure; thereby enabling the manufactured end effector with the target structure to effectively improve the task execution efficiency and success rate, and the feedback compensation mechanism plays a supplementary support role in the process of the target actuator executing the interaction task. The feedback compensation mechanism makes the target actuator more stable and adaptable when executing the task, can timely adjust the interaction actions of the target actuator, maintain the normal progress of the interaction task, greatly improves the robustness of the interaction behavior strategy, avoids the interruption or failure of the interaction task, and thus improves the execution effect of the end effector executing the interaction task. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0011] Figure 1 is one of the flow diagrams of the end effector execution method provided by an embodiment of the present application; Figure 2 is another flow diagram of the end effector execution method provided by an embodiment of the present application; Figure 3 is a structural diagram of Case 1 of the end effector structure in a simulation environment provided by an embodiment of the present application; Figure 4 is a structural diagram of Case 2 of the end effector structure in a simulation environment provided by an embodiment of the present application; Figure 5 It is a schematic diagram of the structure of the end effector in the simulation environment provided by the embodiment of the present application, which is Case 3; Figure 6 It is a schematic diagram of three interaction strategies of the end effector structure Case 1 provided by the embodiment of the present application for a sphere; Figure 7 It is a schematic diagram of two installation application methods of the end effector structure Case 1 provided by the embodiment of the present application; Figure 8 It is an interaction application method of the end effector structure Case 3 provided by the embodiment of the present application; Figure 9 It is a schematic diagram of the structure of the end effector execution device provided by the embodiment of the present application; Figure 10 It is a schematic diagram of the structure of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0012] In order to more clearly understand the above objects, features, and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0013] In the following description, many specific details are set forth to fully understand the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0014] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article, or device. Without further limitation, the element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article, or device comprising the element.

[0015] To solve the problems existing in the related art, the embodiments of the present application provide an end effector execution method, device, and medium.

[0016] First, the end effector execution method provided by the embodiments of the present application will be introduced below. As Figure 1As shown in the figure, the method specifically includes the following steps: S100. Construct a world model according to the environmental information of the end effector performing the target interaction task; the world model is used to characterize the physical characteristics and dynamic characteristics of the interaction environment in which the end effector performs the target interaction task.

[0017] Optionally, in the embodiments of the present application, the end effector is a component that directly performs tasks on a robot or automation device, similar to a human hand or tool, and is a key component for realizing specific operations. The end effector directly interacts with the operation object. For example, in industrial production, it is used to grasp and transport parts, and in home services, it is used to clean and pick up items. The performance and design of the end effector directly affect the effect of task execution, and different task requirements require different types and functions of end effectors.

[0018] The target interaction task refers to the specific task goal that the end effector needs to complete and is the core guidance of the entire operation process. For example, in a mass customization chemical plant, the target interaction task can be to precisely assemble a specific workpiece; in a home service scenario, it can be to clean the room floor and organize items; in a field exploration, it can be to collect specific samples, etc. These tasks have clear expected results, driving a series of subsequent operations and decisions of the end effector.

[0019] The environmental information refers to various information of the environment in which the end effector performs the target interaction task. The scope of these information is wide, including object information in the interaction environment, such as the position, shape, size, material, etc. of the object; the physical conditions of the environment, such as temperature, humidity, light intensity, etc.; and the spatial layout of the environment, such as whether there are obstacles and the width of the passage. At the same time, it also includes relevant information of the interaction object of the interaction task and the actuator body of the end effector, such as the physical properties of the interaction object and the position and posture of the end effector itself. These information are the basic data for constructing the world model.

[0020] The world model is constructed based on the collected environmental information and is used to characterize the interaction environment. It is a digital and abstract representation of the real interaction environment, integrating various elements and relationships in the environment. By constructing the world model, the end effector can "understand" the environment it is in and provide a basis for subsequent decisions. This world model not only contains the static information of the environment but also can describe the dynamic changes of the environment, enabling the end effector to predict the evolution of the environment and make adaptive decisions in advance.

[0021] The interaction environment refers to the specific environmental scope involved when the end effector performs the target interaction task. It can be a factory workshop, a house, a field area, etc. The interaction environment includes various objects, spaces, and conditions related to the task, and is the place where the end effector interacts with the operation object. Its complexity and characteristics directly affect the difficulty and method of the end effector's task execution.

[0022] Physical properties mainly refer to the physical properties of the objects in the interaction environment and the environment itself. For example, the density, hardness, elastic modulus, thermal conductivity of the object, etc. These properties determine the behavior of the object under forces, heat, etc.; the physical properties of the environment also include temperature, humidity, air pressure, etc. These factors will affect the performance of the end effector and the interaction process with the object. For example, high temperature may affect the mechanical properties of the end effector made of certain materials, and a humid environment may cause electronic component failures, etc.

[0023] Dynamic characteristics describe the characteristics of the interaction environment and the objects in it that change over time. It includes the motion state of the object, such as speed, acceleration, and motion trajectory; the dynamic changes of environmental conditions, such as the rise and fall of temperature, and the change of light; as well as the dynamic response during the interaction process between the end effector and the object, such as the change of contact force and the deformation of the object. Understanding the dynamic characteristics helps the end effector adjust its actions and strategies in a timely manner to adapt to the dynamic requirements of the environment and the task, and ensure the smooth execution of the task.

[0024] Optionally, in a feasible implementation manner of this application, in the data acquisition link, ultrasonic sensors can be used to assist in detecting distances, and inertial measurement units can be used to obtain the attitude information of the end effector; in terms of data processing, clustering algorithms in machine learning can be used to classify and integrate the data, or autoencoders in deep learning can be used to automatically extract features; in the model construction stage, knowledge graph technology can be introduced to structurally represent various types of knowledge and relationships in the environment, making the world model more capable of semantic understanding, or combined with virtual reality / augmented reality technology to visually assist in constructing and verifying the world model, enhancing the intuitiveness and accuracy of the model.

[0025] S200, based on the first preset model, determine the target structure of the end effector and the interaction behavior strategy for the end effector to perform the target interaction task according to the world model and the task requirements of the end effector; the first preset model is trained with the goal of maximizing the execution success rate of the target interaction task.

[0026] Optionally, in the embodiments of the present application, the first preset model is a model constructed based on deep learning and is trained with the goal of maximizing the execution success rate of the target interaction task. Through training and learning with a large amount of data, it can analyze the environmental information provided by the world model and combine the task requirements of the end effector to output appropriate decision results.

[0027] Task requirements refer to the specific requirements and expected results that the end effector needs to achieve when performing the target interaction task. It includes the type of task (such as grasping, assembling, handling, etc.), the precision requirements for task execution, the time limit for completing the task, specific constraints of the object to be operated (such as gentle operation for fragile items), and other contents. Task requirements are the key to driving the entire operation process of the end effector, clarifying the direction of the end effector's efforts, and are also important input information for the first preset model to make decisions.

[0028] The target structure is the structural solution of the end effector determined by the first preset model according to the world model and task requirements. This structure covers aspects such as the shape, size, material distribution, and joint configuration of the end effector, aiming to meet the kinematic reachability requirements during task execution and possess appropriate mechanical stiffness and compliance. For example, in a precision assembly task, the target structure may be designed to have geometric adaptability to ensure the robustness of the operation; while in a heavy object handling task, the target structure needs to have sufficient strength and stability.

[0029] The interaction behavior strategy is a set of actions and decisions adopted by the end effector when performing the target interaction task. The interaction behavior strategy includes a basic action sequence, that is, the specific action steps and sequence required for the end effector to complete the task, such as approaching the object, grasping, moving, placing, etc.; it also includes a safety policy library for dealing with possible risk situations during task execution, such as preventing collisions and avoiding overload. The interaction behavior strategy ensures that the end effector can interact with the object to be operated and the environment safely and effectively to achieve the task goal.

[0030] The execution success rate is an indicator to measure the success degree of the end effector in performing the target interaction task and is used to evaluate the effectiveness of the decision-making of the first preset model. It represents the proportion of times the end effector successfully completes the task when performing the same or similar tasks multiple times. The determination basis for successfully completing the task can be set according to the task requirements, such as accurately completing the assembly or completing the handling within the specified time. The first preset model aims to improve this indicator through continuous optimization training, enabling the end effector to complete the work more reliably under various environmental and task conditions.

[0031] Optionally, in a feasible implementation manner of this application, the world model and the task requirements of the end effector are input into the first preset model. The first preset model will deeply analyze and process the input information. The first preset model will evaluate the feasibility and effectiveness of different structures and behavior strategies in this interaction environment according to the environmental conditions reflected by the world model. For example, if the environmental space is narrow, the first preset model may tend to select a smaller and more flexible target structure; if the surface of the object to be operated is smooth, the first preset model will design an interaction behavior strategy more suitable for grasping smooth objects. In determining the target structure, the first preset model will comprehensively consider the requirements of the task for the structure, such as load-bearing capacity, operation accuracy, etc., and the limiting factors of the environment in the world model, and generate the end effector structure scheme that best meets the maximization of the execution success rate, including the specific design of the shape, size, material, etc. of the structure.

[0032] For the interaction behavior strategy, the first preset model will plan a series of action steps and decision rules. These rules cover the entire process from approaching the object to be operated, implementing the operation to completing the task. Finally, the first preset model outputs the determined target structure of the end effector and the interaction behavior strategy for executing the target interaction task, providing a key basis for the subsequent steps.

[0033] S300, based on the second preset model, according to the target structure, determine the structure manufacturing instruction for the end effector to manufacture the target structure, and the quality control rule for the end effector to manufacture the target structure; and according to the interaction behavior strategy, determine the interaction motion control instruction for the end effector to execute the interaction behavior strategy, and the feedback compensation mechanism for the end effector to execute the target interaction task according to the interaction motion control instruction; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interaction robustness of executing the target interaction task.

[0034] Optionally, in the embodiments of this application, the second preset model takes the target structure and the interaction behavior strategy, etc. as inputs, and outputs the structure manufacturing instruction, the quality control rule, the interaction motion control instruction, the feedback compensation mechanism, etc., providing guidance for the manufacturing and motion control of the end effector.

[0035] The structure manufacturing instruction is a series of instructions determined according to the target structure, used to guide the end effector on how to manufacture the target structure, including specific operation steps, action sequences, process parameters, etc., clarifying the specific behavior of the end effector during the manufacturing process.

[0036] Quality control rules are rules formulated to ensure that the manufactured target structure meets certain quality standards. Quality control rules involve monitoring and adjusting various parameters in the manufacturing process, such as material selection, control of processing accuracy, adjustment of manufacturing speed, etc., to ensure the quality stability and reliability of the target structure.

[0037] The interactive motion control instruction is an instruction generated according to the interactive behavior strategy, which is used to control the motion mode of the end effector when performing the target interaction task, including control information such as the motion trajectory, speed, acceleration, and posture, so that the end effector can interact with the environment according to the predetermined interactive behavior strategy.

[0038] The feedback compensation mechanism is a mechanism that adjusts and compensates the motion of the end effector based on feedback information. During the execution of the target interaction task, the difference information between the actual motion state and the expected motion state is obtained through devices such as sensors, and then the interactive motion control instruction is adjusted according to these feedback information to correct the deviation and improve the accuracy and stability of the task execution.

[0039] The structural quality refers to the quality characteristics of the target structure, including aspects such as the strength, stiffness, accuracy, and stability of the structure. A high-quality target structure can better meet the task requirements, withstand the corresponding external forces and environmental impacts, and ensure the reliability and performance of the end effector during the task execution process.

[0040] The interactive robustness of executing the target interaction task represents the ability of the end effector to face various uncertain factors and interferences when performing the target interaction task. Having higher interactive robustness means that the end effector can still maintain stable performance and a high task execution success rate under different environmental conditions, task changes, and possible interferences, and can flexibly adapt to various situations and accurately complete the interaction task.

[0041] Optionally, in a feasible implementation manner of this application, the determined target structure is input into the second preset model. Based on this information, the second preset model combines the knowledge and experience learned during its training, and considers various factors in the manufacturing process, such as the feasibility of the processing technology and the performance limitations of the equipment, to generate a structural manufacturing instruction for the end effector to manufacture the target structure, guiding the end effector on how to construct the target structure step by step.

[0042] Meanwhile, to ensure the quality of the target structure, the second preset model also formulates corresponding quality control rules based on the characteristics and requirements of the target structure. This may involve regulations on aspects such as the quality control of materials during the manufacturing process, the precision requirements for key dimensions and shapes, the inspection methods and standards during the manufacturing process, etc. By strictly following these quality control rules, it can be ensured that the manufactured target structure meets the predetermined quality standards and has good performance and reliability.

[0043] Next, the interaction behavior strategy is input into the second preset model. Based on this information, the second preset model considers various situations that may occur during the task execution process, such as environmental changes, uncertainties of the operation object, etc., and determines the interaction motion control instructions for the end effector to execute the interaction behavior strategy. These instructions precisely control the motion mode of the end effector, including speed, acceleration, attitude adjustment, etc., enabling the end effector to accurately interact with the environment according to the interaction behavior strategy.

[0044] Finally, to address the possible deviations and interferences that may occur during the execution of the interaction motion control instructions, the second preset model formulates a feedback compensation mechanism based on the interaction behavior strategy and possible error situations to ensure that the end effector can stably and accurately execute the target interaction task.

[0045] S400, according to the world model, the target structure, the structure manufacturing instruction, and the quality control rule, control the end effector to manufacture the target structure in real-time on-demand through the multi-process additive manufacturing function; S500, when the manufacturing of the target structure is completed, according to the interaction behavior strategy, the interaction motion control instruction, and the feedback compensation mechanism, control the target actuator to execute the target interaction task, where the target actuator is the end effector with the target structure.

[0046] Optionally, in the embodiment of the present application, the target actuator refers to the actuator formed by adding a target structure with a specific design through the real-time additive manufacturing function to the basic end effector. The basic end effector is the initial state actuator without structure manufacturing, and the basic end effector has basic motion and operation functions but has not been structurally optimized for specific interaction tasks. The target structure manufactured in real-time through the multi-process additive manufacturing function is a dedicated functional module designed according to the task requirements. After combining the target structure with the basic end effector, a target actuator is formed, enabling the target actuator to have kinematic reachability, stiffness, and compliance required for the interaction task in terms of structure, and supporting specific interaction behavior strategies in terms of function, thus significantly improving the task execution ability.

[0047] Optionally, in a feasible implementation manner of the present application, the structure manufacturing instruction, the quality control rule, the interactive motion control instruction, and the feedback compensation mechanism act synergistically. The structure manufacturing instruction guides the multi-process additive manufacturing module to complete the manufacturing of the target structure, and the quality control rule ensures that the manufactured structure meets the quality standards. The interactive motion control instruction accurately controls the motion trajectory, speed, and posture of the end effector according to the interactive behavior strategy. The feedback compensation mechanism monitors the motion state and interactive situation of the target actuator in real time. Once deviations such as collisions and slips occur, it immediately adjusts the interactive motion control instruction to correct the deviation. During the whole process, each element cooperates with each other to dynamically adjust the actions of the target actuator, enabling it to accurately complete the target interactive task and improving the success rate of task execution.

[0048] In a method for an end effector to execute a target interactive task provided in an embodiment of the present application, by constructing a world model to comprehensively represent the physical and dynamic characteristics of the interactive environment, it can accurately reflect the actual situation of the interactive environment. Based on a first preset model trained with the goal of maximizing the success rate of task execution, the target structure and the interactive behavior strategy are determined in combination with the world model and the task requirements. A second preset model trained with the goal of improving the structure quality and interactive robustness determines the structure manufacturing instruction, the quality control rule, the interactive motion control instruction, and the feedback compensation mechanism. Thus, when manufacturing the target structure, the quality control rule ensures the reliability of the target structure; thereby enabling the end effector with the target structure to be manufactured to effectively improve the task execution efficiency and success rate, and the feedback compensation mechanism plays a supplementary support role during the process of the target actuator executing the interactive task. The feedback compensation mechanism makes the target actuator more stable and adaptable when executing the task, can timely adjust the interactive actions of the target actuator, maintain the normal progress of the interactive task, greatly improves the robustness of the interactive behavior strategy, avoids the interruption or failure of the interactive task, and thus improves the execution effect of the end effector executing the interactive task.

[0049] In one embodiment, constructing the world model according to the environmental information of the end effector executing the target interactive task includes: Obtain the environmental information, where the environmental information includes the interactive environment, the interactive object of the interactive task, and the geometric and physical information of the actuator body of the end effector; Perform time calibration and feature extraction on the environmental information to obtain target feature information; Construct the world model according to the target feature information, and the world model is used to realize the dynamic perception and prediction of the interactive environment.

[0050] Optionally, in the embodiments of the present application, the interaction object refers to an object that directly interacts with the end effector during the execution of the target interaction task. In different task scenarios, the interaction object will be different. In an industrial manufacturing scenario, it may be the parts to be processed or the components to be assembled; in a home service scenario, it may be the furniture to be cleaned or the items to be moved; in a field exploration scenario, it may be the plants or geological rocks from which samples need to be collected. The characteristics of these interaction objects, such as shape, material, weight, etc., will have an important impact on the operation methods and strategies of the end effector.

[0051] Geometric information mainly describes the spatial characteristics of an object, such as shape, size, position, and posture. For the interaction environment, geometric information can include the contours and layouts of objects in the environment, as well as the size and shape of the space; for the interaction object, geometric information is reflected in its specific shape, whether it is a regular shape or an irregular shape, and its size; for the actuator body of the end effector, geometric information involves the shapes and sizes of its various components, as well as their positions and postures in space, and this information affects the interaction method and reachable range between the end effector and the interaction object.

[0052] Physical information includes various physical properties of an object. For the interaction environment, physical information includes environmental factors such as temperature, humidity, light intensity, and gravitational field; for the interaction object, it includes material (such as metal, plastic, wood, etc.), density, hardness, elastic modulus, friction coefficient, etc., and these properties determine the performance of the interaction object under forces, heat, etc.; for the actuator body of the end effector, physical information covers mass, inertia, motor torque, joint stiffness, etc., and this information is related to the motion performance and operation ability of the end effector. In these optional embodiments, obtaining the geometric and physical information of the interaction environment, interaction object, and the actuator body of the end effector provides basic data for comprehensively understanding the interaction environment, enabling the end effector to perceive the surrounding environment and its own status. Time calibration and feature extraction further optimize the data, eliminate the spatio-temporal deviation of different sensor data, and extract key target feature information, improving the data quality and usability. Finally, constructing a world model based on the target feature information can accurately represent the physical and dynamic characteristics of the interaction environment, providing a reliable basis for the end effector to subsequently determine the target structure, interaction behavior strategy, etc., enhancing the adaptability and decision-making accuracy of the end effector in complex task scenarios and contributing to improving the task execution success rate.

[0053] In one embodiment, the time calibration and feature extraction of the environmental information to obtain target feature information includes: Establishing a unified coordinate system for the environmental information collected from different acquisition channels based on a three-dimensional space registration algorithm to obtain the first information; Time-calibrate the first information by combining a distributed timestamp with a least squares algorithm to obtain calibrated information; Extract features from the calibrated information to obtain the target feature information.

[0054] In these alternative embodiments, a unified coordinate system is established based on a three-dimensional spatial registration algorithm, and environmental information from different acquisition channels is integrated into the same spatial framework, avoiding information confusion caused by coordinate system differences, ensuring the consistency of the spatial positions of the information, and facilitating subsequent comprehensive analysis. The combination of a distributed timestamp and a least squares algorithm is used for time calibration to eliminate the time deviation of the information, synchronize the information in the time dimension, and ensure the timeliness and coherence of the data. Feature extraction is performed on the calibrated information to extract key target feature information from complex data, remove redundancy, highlight useful information, and provide strong support for constructing an accurate world model, thereby assisting the end effector to complete tasks more efficiently.

[0055] It should be noted that the above processing methods based on the three-dimensional spatial registration algorithm and the combination of the distributed timestamp and the least squares algorithm are only for illustrative purposes. It should be emphasized that this application is not limited to the specific technical methods adopted. No matter what technical means (such as using Kalman filtering to calibrate the time deviation, a spatio-temporal feature alignment algorithm based on deep learning, etc.), as long as it can achieve time calibration and alignment of environmental information, eliminate the differences in the spatial coordinate system and time dimension of different acquisition channels, and ensure the consistency, synchronization, and accuracy of environmental information in the spatio-temporal dimension, it can be used to construct the world model.

[0056] In one embodiment, the extracting features from the calibrated information to obtain the target feature information includes: When the calibrated information is geometric point cloud data, perform feature extraction on the geometric point cloud data by using the principal component analysis method to obtain the eigenvalues corresponding to the geometric point cloud data; When the calibrated information is the motion pose sequence of the end effector, extract the joint friction coefficients of the motion pose sequence; When the calibrated information is temperature field data, extract the core time frequency information of the temperature fluctuation in the temperature field data by using Fourier transform; Wherein, the target feature information includes the eigenvalues, the joint friction coefficients, and the core time frequency information.

[0057] Optionally, in the embodiments of the present application, geometric point cloud data is a set of discrete points used to represent a three-dimensional object or scene. It is collected by devices such as lidar and depth cameras. Each point contains three-dimensional spatial coordinates (X, Y, Z), and may also contain information such as color and reflection intensity. These points can approximately describe the surface shape of the object and are used in the present application to present the geometric form of the interaction environment, interaction object, or the actuator body of the end effector.

[0058] The eigenvalue measures the degree of dispersion of the data in the direction of the corresponding principal component. The larger the value, the more original data information is contained in this direction, and it can reflect the key geometric features of the point cloud data.

[0059] The motion pose sequence records the position and pose information of the end effector at a series of moments during the motion process. The position refers to the coordinates in three-dimensional space, and the pose can be represented in ways such as rotation matrix, Euler angles, or quaternions, which describe the orientation of the end effector. The motion pose sequence can completely present the motion trajectory and pose change process of the end effector.

[0060] When the joints of the end effector move, due to the contact and relative motion between the joint components, friction is generated. The joint friction coefficient is a proportional coefficient that measures the magnitude of this frictional force. It affects the smoothness of joint motion, energy consumption, and the accuracy of motion control, and is an important parameter reflecting the mechanical characteristics of the end effector.

[0061] The temperature field data is a set of data describing the temperature distribution in space. Within a certain spatial range, the temperature values at different positions and the change of temperature over time constitute the temperature field data. It can be collected through a temperature sensor network, etc., and is used to characterize the temperature state of the interaction environment, the actuator body of the end effector, or the interaction object.

[0062] For the temperature field data, the Fourier transform is used to convert the temperature fluctuation signal in the time domain to the frequency domain for analysis. Temperature fluctuations often contain multiple frequency components. The core time frequency information refers to the frequency components with a relatively large proportion of energy and playing a major role in temperature changes, reflecting the main periodic characteristics and change laws of temperature fluctuations.

[0063] It should be noted that in the above embodiments, operations such as using the principal component analysis method for geometric point cloud data, extracting the joint friction coefficient from the motion pose sequence, and processing the temperature field data through Fourier transform are only illustrative examples of feature extraction. It should be clear that the present application is not limited to the above specific data types and extraction methods. Whether it is data such as pressure distribution, light intensity in the environment, or vibration frequency, current consumption of the end effector, as long as key features related to the interaction environment and the state of the end effector can be extracted from the calibration information through reasonable algorithms and technical means, they are applicable to the present application.

[0064] Optionally, in a specific embodiment of the present application, as Figure 2 shown, the end effector of the present application includes three major modules: a multi-modal perception module, a multi-process additive manufacturing module, and an embodied intelligent agent module based on deep learning.

[0065] Optionally, in a specific implementation manner, first is the link of obtaining environmental information. In the actual operation scenario, various sensors in the multi-modal perception module are used to collect information. A depth camera or a laser scanner is used to collect geometric information such as the shape, size, color, and texture of the interaction environment and the interaction object of the interaction task, which is presented in the form of point cloud data, such as , which contains the three-dimensional coordinates and color information of each point. A temperature sensor is used to obtain the temperature field distribution of the interaction environment and the interaction object, like describes the temperature conditions at different positions in space at time t. The actuator body of the end effector is driven by the data exchange system of the multi-process additive manufacturing module to collect its dynamic characteristics in the on-site environment, for example, obtaining time series pose data . The body can also be driven to touch the operation object, and the mechanical properties of the operation object, such as the stress-strain relationship , are collected by means of a tactile sensor, so that the environmental information covering the geometric and physical information of the interaction environment, the interaction object, and the actuator body of the end effector is obtained.

[0066] Then, time calibration and feature extraction are carried out. For the multi-source heterogeneous environmental information collected, first, based on the three-dimensional space registration algorithm, the information obtained from different acquisition channels is established in a unified coordinate system, and the data under different spatial references is integrated into the same framework to obtain the first information. Then, the distributed time stamp is combined with the least square algorithm to perform time calibration on the first information to eliminate the time deviation caused by factors such as different sensor sampling times, and the calibrated information is obtained. After that, feature extraction is carried out for the calibrated information. If the calibrated information is geometric point cloud data, the principal component analysis method is used to find the direction with the largest data variance and obtain the corresponding eigenvalues, such as , and these eigenvalues reflect the key geometric features of the point cloud data; when the calibrated information is the motion pose sequence of the end effector, the joint friction coefficient is extracted to reflect the friction characteristics of joint movement; if the calibrated information is temperature field data, the temperature fluctuation signal in the time domain is converted to the frequency domain through Fourier transform, and the core time frequency information of the temperature fluctuation is extracted to reflect the main periodic characteristics of temperature change. Through these operations, the target feature information is obtained.

[0067] In these optionally implemented embodiments, principal component analysis processes geometric point cloud data, extracts eigenvalue, can effectively compress the data dimension, highlight key geometric features, and facilitate the distinction of object shape structures. Extracting the joint friction coefficients of the motion pose sequence can accurately reflect the motion characteristics of the end effector joints, and help optimize motion control and energy consumption management. Fourier transform extracts the core time-frequency information from the temperature field data, reveals the temperature fluctuation law, and provides a basis for decisions related to the ambient temperature. In summary, the target feature information obtained through these steps lays a solid foundation for constructing an accurate world model and improving the task execution ability of the end effector.

[0068] In one embodiment, constructing the world model according to the target feature information includes: Reconstructing a three-dimensional geometric model of the interaction environment through implicit neural representation according to the target feature information corresponding to the interaction environment; Constructing a multi-physical property model of the interaction object through a material constitutive equation according to the target feature information corresponding to the interaction object; Determining the body dynamics model of the actuator body through a Lie group motion chain according to the target feature information corresponding to the actuator body; Determining the interaction dynamics model between the actuator body and the interaction environment through Hamilton's equation according to the body dynamics model; Constructing an energy potential contact model describing the contact between the actuator body and the interaction object according to the target feature information; Constructing a state transition equation predicting the multi-physical field evolution of the interaction environment according to the target feature information; Constructing the world model according to the three-dimensional geometric model, the multi-physical property model, the body dynamics model, the interaction dynamics model, the energy potential contact model, and the state transition equation.

[0069] Optionally, in the embodiments of the present application, implicit neural representation is a method of representing geometric shapes or other continuous signals using a neural network. It can learn the latent structure in the data and map the input target feature information to a continuous function. When constructing a three-dimensional geometric model of the interaction environment, through implicit neural representation, the powerful fitting ability of the neural network can be utilized to learn the geometric shapes, surface features, etc. of the objects in the environment according to the target feature information corresponding to the environment, so as to reconstruct the three-dimensional geometric model.

[0070] The three-dimensional geometric model is a mathematical description of the three-dimensional spatial structure of the interaction environment. It represents the shape, size, and positional relationships of objects in the environment through geometric elements such as points, lines, and planes. In this embodiment, it is constructed based on the target feature information corresponding to the interaction environment using the implicit neural representation method, providing a spatial basis for subsequent analysis of the movement and interaction of the end effector in this environment.

[0071] The body dynamics model describes the relationship between the movement and mechanical characteristics of the end effector's actuator body. Through the Lie group motion chain, combined with the target feature information corresponding to the end effector's actuator body (such as joint friction coefficients, etc.), the motion states, force conditions, and their interactions of each joint of the end effector can be determined, thereby establishing a body dynamics model for predicting and controlling the movement of the end effector.

[0072] The interaction dynamics model studies the dynamic relationship of the interaction between the end effector's actuator body and the interaction environment. Based on the body dynamics model, through the Hamiltonian equation, considering factors such as force transfer and energy exchange between the end effector and the environment, the dynamic behaviors of the end effector and the environment during the interaction are determined, helping to understand and optimize the operation of the end effector in the environment.

[0073] The energy potential contact model is used to describe the energy and potential energy changes when the end effector's actuator body comes into contact with the interaction object. According to the target feature information, considering factors such as force, deformation, and energy conversion during contact, this model is constructed, which can analyze the mechanical characteristics and energy distribution during the contact process, providing a basis for designing reasonable contact strategies and evaluating contact effects.

[0074] Multi-physical field evolution refers to the change process of multiple physical fields (such as temperature field, electromagnetic field, stress field, etc.) in the interaction environment over time. These physical fields affect and couple with each other, jointly determining the dynamic characteristics of the environment.

[0075] The state transition equation is a mathematical equation used to predict the multi-physical field evolution of the interaction environment. Through this state transition equation, the dynamic changes of the environment can be predicted, enabling the end effector to make corresponding decisions and adjustments in advance.

[0076] Optionally, in a specific implementation, first, for the construction of the three-dimensional geometric model of the interaction environment. Based on the target feature information corresponding to the interaction environment, the implicit neural representation method can be used. Taking these feature information as the input of the neural network, using the powerful non-linear fitting ability of the neural network, by continuously training the network, it enables the network to learn the geometric shapes, spatial layouts, etc. of the objects in the interaction environment. The continuous function output by the neural network can describe any point in the environment, thereby reconstructing the three-dimensional geometric model of the interaction environment and accurately presenting the spatial structure of the environment.

[0077] Next, for the construction of the multi - physical property model of the interaction object. Based on the target feature information corresponding to the interaction object, combined with the material constitutive equation. The material constitutive equation describes the relationships between physical quantities such as stress, strain, and temperature of the material. Substitute the parameters related to the material (such as material type, physical properties, etc.) in the target feature information into the corresponding constitutive equation, and through calculation and analysis, determine the multi - physical properties such as mechanics and thermotics of the interaction object under different conditions, and then construct a multi - physical property model that can accurately reflect the physical characteristics of the interaction object.

[0078] Then, determine the kinematic model of the end - effector body. With the help of the target feature information corresponding to the end - effector body, adopt the Lie group kinematic chain. The Lie group kinematic chain is a mathematical tool for describing robot kinematics. By modeling the motion of each joint of the end - effector and combining parameters such as joint friction coefficients in the target feature information, it can accurately determine the motion state, force condition, etc. of the end - effector body, thus establishing a kinematic model to achieve accurate description of the motion of the end - effector body.

[0079] After that, construct the interaction kinematic model. Based on the established kinematic model, use the Hamiltonian equation. The Hamiltonian equation is an important equation in analytical mechanics, which describes the dynamic behavior of the system from the perspective of energy. Substitute the relevant parameters in the kinematic model into the Hamiltonian equation, consider factors such as energy exchange and force transfer between the end - effector body and the interaction environment, calculate the dynamic relationship between the two during the interaction process, and then determine the interaction kinematic model.

[0080] Then, construct the energy - potential contact model. According to the target feature information, consider factors such as force, deformation, and energy conversion when the end - effector body contacts the interaction object. By establishing a mathematical model to describe the change laws of energy and potential energy during the contact process, simulate the contact kinematics, and thus construct an energy - potential contact model for analyzing the mechanical characteristics and energy distribution during contact.

[0081] Next, construct the state - transfer equation. Based on the target feature information, consider the change laws of multi - physical fields (such as temperature field, stress field, etc.) in the interaction environment. By analyzing the historical data and current state of these physical fields, establish an equation that can describe the relationship between the current state and the future state, so as to predict the evolution of the multi - physical fields in the interaction environment.

[0082] Finally, integrate the three-dimensional geometric model, multi-physical property model, body dynamics model, interaction dynamics model, energy potential contact model, and state transition equation constructed above. These models and equations describe the characteristics and interrelationships of the interaction environment, interaction objects, and the actuator body of the end effector from different perspectives. By synthesizing this information, a complete world model is constructed to provide comprehensive and accurate basic support for the end effector to perform the target interaction task.

[0083] Optionally, in the above embodiments, operations such as reconstructing the three-dimensional geometric model through implicit neural representation and constructing the multi-physical property model using the material constitutive equation are only exemplary technical means for constructing the world model. It should be emphasized that this application is not limited to the above specific feature extraction dimensions and modeling methods. Whether using a topology optimization algorithm to generate an environmental model or relying on machine learning to predict the behavior of interaction objects, as long as it can effectively represent the physical characteristics of the interaction environment, the property changes of interaction objects, the dynamic behavior of the end effector, and the dynamic interaction relationship among the three based on the target feature information, it can be used to construct the world model.

[0084] In these optional embodiments, the three-dimensional geometric model is reconstructed by implicit neural representation to accurately restore the spatial structure of the interaction environment; the multi-physical property model is constructed by the material constitutive equation to clarify the physical characteristics of the interaction object; the dynamic model is determined by the Lie group kinematic chain and the Hamiltonian equation to master the motion and interaction laws between the actuator and the environment; the energy potential contact model analyzes the contact characteristics, and the state transition equation predicts environmental changes. The world model constructed by integrating these models provides a reliable basis for the decision-making and control of the end effector in complex tasks, improving the accuracy and efficiency of task execution.

[0085] In one embodiment, the first preset model is trained with the goal of maximizing the execution success rate of the target interaction task, and includes: Construct a first initial model that takes into account the environmental constraints of the interaction environment and the diversity of task requirements of the interaction task; Train the first initial model with the goal of maximizing the task execution success rate through the first preset training samples until the first preset training stop condition is met to obtain the first preset model; the first preset model is used to determine the target structure and interaction behavior strategy, and the structural features of the target structure include the kinematic reachability, mechanism stiffness, and compliance of the target structure; the interaction behavior strategy includes a basic action sequence and a safety policy library; The second preset model is trained with the goal of maximizing the structural quality of the target structure and the interaction robustness of performing the target interaction task, and includes: Construct a second initial model; Using the second preset training samples, the second initial model is trained with the goal of maximizing the structural quality of the target structure and the interaction robustness for performing the target interaction task until the second preset training stop condition is met, obtaining the second preset model; wherein, when the second preset model determines the structural manufacturing instruction and the quality control rule, path continuity and obstacle avoidance are considered; when the second preset model determines the interaction motion control instruction and the feedback compensation mechanism, different failure modes of the end effector are considered.

[0086] Optionally, in the embodiments of the present application, the environmental constraints of the interaction environment refer to the constraints imposed by the interaction environment on the end effector and the task implementation during the execution of the interaction task. For example, when operating in a narrow space, the space size will limit the movement range of the end effector; in an environment with high temperature, high humidity or strong electromagnetic interference, special environmental factors may affect the performance of the electronic components and material properties of the end effector, thus hindering the task execution.

[0087] The diversity of task requirements means that different interaction tasks have various requirements. For example, in industrial production, assembly tasks have extremely high precision requirements, and the end effector needs to operate on parts accurately; while handling tasks pay more attention to load-bearing capacity and movement efficiency; in the home service scenario, cleaning tasks require the end effector to be able to adapt to the surface materials and shapes of different furniture for cleaning. These different requirements of different tasks reflect the diversity of task requirements.

[0088] Kinematical reachability describes the ability of the end effector to reach positions and postures in space. A target structure with good kinematical reachability means that the end effector can freely move to the required position in the task space, adjust to the appropriate posture, and complete various actions, such as grasping and placing objects, without being restricted by its own structure.

[0089] The mechanism stiffness reflects the ability of the end effector structure to resist deformation. When it is necessary to bear large external forces or perform high-precision operations, sufficient mechanism stiffness can ensure the stability of the end effector structure, maintain an accurate motion trajectory and operation accuracy, and avoid excessive deformation due to force application, which may affect the task execution.

[0090] The mechanism compliance is opposite to the stiffness, referring to the characteristic that the end effector structure can adapt to external forces and undergo certain deformation. When interacting with fragile objects or when it is necessary to adapt to the surface of complex-shaped objects, compliance can prevent the end effector from damaging the object and better conform to the object surface to achieve stable operation.

[0091] The basic motion sequence is an ordered combination of a series of basic actions performed by the end effector to complete the target interaction task. Taking the grasping task as an example, the basic motion sequence may include steps such as approaching the object, adjusting the grasping posture, performing the grasping action, and lifting the object. These actions are executed in a certain order and are the basic processes to complete the task.

[0092] The safety policy library contains a series of rules and policies for ensuring the safe operation of the end effector during the task execution. For example, setting a force threshold, when the detected contact force exceeds the threshold, the end effector stops moving to prevent damage to itself, the interaction object, or the surrounding environment; or formulating emergency handling strategies in case of emergencies (such as power failure, sensor failure) to ensure the safety of the system.

[0093] The path continuity requires that during the movement of the end effector, its movement path should be continuous and smooth, avoiding sudden jumps or interruptions. This can ensure the stability of the movement, reduce mechanical shocks, improve the movement efficiency, and is also beneficial for precise control of the movement.

[0094] Obstacle avoidance means that during the movement of the end effector, it can detect the obstacles in the surrounding environment in real time and plan a reasonable path to avoid them, preventing collision accidents and ensuring the safe and smooth execution of the task.

[0095] Different fault modes refer to various fault situations that may occur during the operation of the end effector. For example, motor failure leads to abnormal driving of the joints; sensor failure makes the obtained environmental information inaccurate; communication failure causes abnormal transmission of control commands, etc. Considering these fault modes when determining the interaction motion control commands and feedback compensation mechanisms can enable the end effector to take corresponding measures to maintain operation or perform a safe shutdown when a fault occurs, improving the reliability and stability of the system.

[0096] Optionally, in a specific implementation manner of this application, first construct a first initial model, which fully considers the environmental limitations of the interaction environment and the diversity of task requirements of the interaction task. When constructing the model, use the deep learning method to build a framework with a specific network structure to lay a foundation for subsequent learning. Then, use the first preset training sample to train the first initial model, with the goal of maximizing the execution success rate of the target interaction task. The training process is based on the objective function: ; as the optimization direction, where, 、 are weight coefficients, measures the deviation between the actual position of the end effector and the target position , Represents the magnitude of the contact force. By continuously adjusting the model parameters, the model learns how to generate optimal target structures and interaction behavior strategies according to different environmental and task situations.

[0097] Regarding the target structure, kinematic reachability, mechanism stiffness, and compliance need to be comprehensively considered. When the first preset training stop condition is met, such as when the loss function of the model converges to a certain extent or the accuracy no longer improves, the first preset model is obtained. The first preset model aims to maximize the task execution success rate and quantifies the evaluation of the task execution success rate as follows: ; where is the task execution success rate, is the interaction behavior strategy generated based on the target structure. Finally, the target structure under the optimized design is obtained, covering geometric configuration, material distribution, kinematic constraints, etc., and the interaction behavior strategy, such as the target pose sequence, expected interaction contact force threshold, safety rule base, etc.

[0098] Construct a second initial model as the basic framework. Use the second preset training samples to train it, aiming to maximize the structural quality of the target structure and the interaction robustness of performing the target interaction task. During the training process, the model learns to generate structure manufacturing instructions and quality control rules, and at this time, path continuity and obstacle avoidance are considered. When determining the interaction motion control instructions and feedback compensation mechanism, the model considers different failure modes of the end effector, such as but not limited to collision, slip, failure, etc. Taking maximizing the part quality and interaction robustness as the goal, factors such as process constraints, environmental interference, and estimation error are considered: ; where represents the structural quality of the target structure, represents the interaction robustness, respectively represent process constraints, environmental interference, and estimation error.

[0099] When the second preset training stop condition is met, the second preset model is obtained. The second preset model generates additive manufacturing, interaction action instructions, and closed-loop feedback control logic, including system motion control instructions, real-time compensation mechanisms, etc., and finally obtains the expected actuator structure entity and completes the interaction task. Through the training and optimization of these two models, from structural design, behavior planning to manufacturing and interaction control, it comprehensively ensures that the end effector can efficiently and safely complete tasks in complex environments.

[0100] In these optionally implemented embodiments, the first preset model takes into account environmental constraints and task requirement diversity, determines the target structure and interaction behavior strategy through training, and can improve the execution success rate of the target interaction task. A reasonable target structure ensures kinematic reachability, mechanism stiffness, and compliance, and a suitable interaction behavior strategy provides a basic action sequence and a safety policy library, enabling the actuator to better adapt to complex tasks. The second preset model focuses on structural quality and interaction robustness, considers paths, obstacles, and failure modes when determining instructions and rules, can ensure the fabrication of high-quality structures, enhance stability and reliability during task execution, reduce the impact of failures, and overall improve the performance and practicality of the end effector.

[0101] In one embodiment, controlling the end effector to fabricate the target structure in real time on demand through a multi-process additive manufacturing function according to the world model, the target structure, the structural fabrication instructions, and the quality regulation rules includes: During the process of the end effector fabricating the target structure according to the structural fabrication instructions, based on the world model, adjust the structural fabrication instructions according to the fabrication state of the end effector at the current moment and the quality regulation rules, so that the fabrication quality of the target structure meets the first preset quality.

[0102] In one embodiment, controlling the target actuator to execute the target interaction task according to the interaction behavior strategy, the interaction motion control instructions, and the feedback compensation mechanism includes: During the process of the target actuator executing the target interaction task according to the interaction motion control instructions, based on the world model, adjust the interaction motion control instructions according to the interaction state of the target actuator at the current moment and the feedback compensation mechanism, so that the execution quality of the end effector executing the target interaction task meets the second preset quality.

[0103] Optionally, as Figure 2 shown, in terms of target structure fabrication, based on the generated structural fabrication instructions, the motion control system and the material supply system of the multi-process additive manufacturing module work together to start executing the pre-planned fabrication instructions to create the target structure. During the fabrication process, the data exchange system real-time feeds back the system state to the world model. The world model combines the fabrication state of the end effector at the current moment, such as the material deposition position, the structural forming progress, etc., and the quality regulation rules to evaluate and analyze the fabrication quality. If it is found that there are factors that may affect the quality during the fabrication process, such as uneven material deposition, structural size deviation, etc., the structural fabrication instructions will be adjusted according to this information. For example, adjust the path planning of the motion control system to make the material deposition more uniform, or change the material supply speed to ensure that the fabrication quality of the target structure meets the first preset quality, achieving efficient and high-quality target structure fabrication.

[0104] During the interactive task execution phase, according to the interactive motion control instructions, the fabricated target actuator is driven to start moving. The multi-modal perception module collects system information in real time, including the position and posture of the target actuator, the contact state with the interaction object, etc., and transmits this information to the world model. The world model evaluates the interactive motion control instructions using the feedback compensation mechanism based on the current interactive state of the target actuator. If an abnormality is detected during the interaction, such as excessive contact force or slipping, the world model will adjust the interactive behavior in real time according to the preset feedback compensation criteria. This may involve adjusting the contact impedance parameters to change the compliance degree when the target actuator contacts the interaction object, or performing reverse displacement compensation to correct the motion deviation of the end effector, ensuring the high-quality execution of the interactive task. At the same time, a task evaluation report is generated to provide data support for the continuous intelligent growth of embodied intelligence and continuously optimize the performance of the target actuator in subsequent tasks.

[0105] In these optionally implemented embodiments, when manufacturing the target structure, based on the world model, the structure manufacturing instructions are adjusted in combination with the manufacturing state and quality control rules, which can correct manufacturing deviations in a timely manner, ensure that the manufacturing quality of the target structure reaches the first preset quality, and avoid affecting subsequent tasks due to manufacturing defects. When performing the interactive task, adjusting the interactive motion control instructions according to the world model, interactive state and feedback compensation mechanism enables the target actuator to quickly adapt to task changes, effectively respond to emergencies, and make the execution quality meet the second preset quality. Generally speaking, these steps improve the adaptability and reliability of the target actuator, ensuring the accurate and efficient completion of the task.

[0106] In a specific embodiment of the present application, an end effector is provided, and its technical solution mainly includes a multi-modal perception module, a multi-process additive manufacturing module, and an embodied intelligent agent based on deep learning. The multi-modal perception module is used to sense the real-time scene situation, collect physical information of the environment and the operation object, and construct a virtual world model of the task scene, which is the decision-making basis for the actuator form design and behavior reasoning; the multi-process additive manufacturing module is responsible for the on-demand on-site manufacturing of the end effector, which is the functional basis for realizing the self-growth of the end effector; the embodied intelligent agent based on deep learning is responsible for generating the required structure according to the task and environmental requirements, reasoning the interaction behavior of the object, and controlling the manufacturing-interaction process with the goal of maximizing the task execution success rate. The present invention realizes the on-demand autonomy of the end effector in various unstructured environments, replaces the traditional task / environment-oriented designed actuator, can effectively solve the challenges of diverse end effectors required under existing unstructured environments and limited resources, expands the feasible environment range of robot compliant manipulation, and improves the task execution success rate.

[0107] The multi-modal perception module constructs a virtual world model required for tasks by collecting on-site data, including information related to the geometric and physical properties of the environment, the manipulated object, and the operating body.

[0108] The multi-process additive manufacturing module forms the required structure by layer-by-layer depositing materials directly on the actuator base, or continues to deposit materials on the existing structure to form the required structure, or continues to deposit materials after partially removing the existing structure to form the required structure. The materials involved include but are not limited to thermoplastics, thermosetting plastics, fiber-reinforced composites, metal / ceramic slurries, liquid metals, metal powders, etc. The manufacturing processes used include but are not limited to fused deposition modeling, direct ink writing, directed energy deposition, etc.

[0109] As Figure 2 shown, it mainly includes three modules: the multi-modal perception module (steps 1-3), the multi-process additive manufacturing module (steps 4-5), and the deep learning-based embodied agent (steps 6-7). The specific implementation is as follows: Step 1: On-site data collection. At the operation site, various sensors equipped in the multi-modal perception system are used to collect the geometric and physical information of the operation environment, the operation object, and the operating body. Specifically, it includes but is not limited to using a depth camera or a laser scanner to collect information such as the shape, size, color, and texture of the environment and the object, as ; using a temperature sensor to collect the temperature field distribution of the environment and the object, as ; driving the movement of the body through the data exchange system of the multi-process additive manufacturing module to collect the dynamic characteristics of the body in the on-site environment, such as time-series pose data ; driving the body to touch the operation object and combining with a tactile sensor to collect the mechanical properties of the operation object, such as the stress-strain relationship etc.

[0110] Step 2: Multi-modal data analysis. For the multi-source heterogeneous data collected in the previous step, the time calibration of the sensor data is performed through distributed time stamps combined with the least squares algorithm, and a unified coordinate system is established based on the three-dimensional space registration algorithm to eliminate the spatio-temporal offset error between different sensors. Extract the corresponding features from the spatio-temporally aligned data, including but not limited to using principal component analysis to extract eigenvalue for geometric point cloud data ; extracting the joint friction coefficient for the body movement pose sequence etc.; extracting the core time frequency information of temperature fluctuations from the temperature field data through Fourier transform ; Step 3: World model construction. Based on the above - collected data and extracted features, an interpretable and predictable multi - physical digital twin of the operation scenario is established, including but not limited to reconstructing the three - dimensional geometry of the environment through implicit neural representation, modeling the multi - physical properties of objects by combining material constitutive equations; describing the kinematics of the body using a Lie group motion chain and embedding the Hamiltonian equation to characterize the environmental interaction dynamics; simulating contact kinematics based on an energy - potential contact model and constructing a state - transfer equation to predict the evolution of multi - physical fields.

[0111] Step 4: Structure design and behavior reasoning. According to the task requirements and environmental constraints, through an autonomous policy network constructed by deep - learning methods, with the objective function: ; Optimize the structure of the end - effector, including but not limited to kinematic reachability, mechanism stiffness, and compliance, and generate interaction behavior strategies , including but not limited to basic action sequences, safety policy libraries, etc.

[0112] Step 5: Generation of manufacturing - interaction process control strategies. Based on the actuator structure generated in the previous step, considering elements including but not limited to path continuity, obstacle avoidance, etc., generate structure manufacturing instructions and online quality control criteria; based on the interaction behavior strategies generated in the previous step, considering failure modes including but not limited to collision, slip, failure, etc., generate interaction motion control instructions and feedback compensation criteria, etc.

[0113] Step 6: Multi - process additive manufacturing. Based on the generated structure manufacturing instructions, drive the motion control system and material supply system of the multi - process additive manufacturing module, execute the pre - planned manufacturing instructions to create the target structure, and through the data exchange system, real - time feedback the system state to the constructed world model for manufacturing quality assessment and analysis, and combine the constructed online quality control criteria to ensure the efficient and high - quality realization of the target structure manufacturing.

[0114] Step 7: Intelligent interaction behavior control. Based on the generated interaction action instructions, drive the motion of the end - effector in online manufacturing. Combine the system information collected in real - time by the multi - modal perception module, evaluate the current state in real - time through the world model and predict the next - stage state, and adjust the interaction behavior in real - time according to the established feedback compensation criteria, including but not limited to adjusting contact impedance parameters, reverse displacement compensation, etc., to ensure the high - quality execution of the interaction task and generate a task evaluation report for the continuous intelligent growth of embodied intelligence.

[0115] As Figure 3 shown, it shows Case 1 of the end - effector structure in the simulation environment.

[0116] Moving joint part: The first Z-axis motion joint 1 and the second Z-axis motion joint 7: They can move the end effector up and down in the Z-axis direction to adjust the vertical height position to meet different operation requirements. For example, when performing additive manufacturing, it can control the distances between the first print head 10 and the first additive manufacturing substrate 9 and the second additive manufacturing substrate 11 respectively, or adjust the height when grasping an object.

[0117] The X-axis motion joint 3: It is responsible for the horizontal movement of the end effector in the X-axis direction, used to adjust the position left and right on the horizontal plane to expand the lateral range of operation.

[0118] The first Y-axis motion joint 8 and the second Y-axis motion joint 12: They enable the horizontal movement of the end effector in the Y-axis direction to adjust the position back and forth. Cooperating with the X-axis motion joint 3, the end effector can be accurately positioned on the plane. Through the coordinated work of the motion joints of these three axes, the end effector can move flexibly in three-dimensional space to reach the target position.

[0119] Sensing components: The first depth camera 2: It can obtain the depth information of the surrounding environment and objects, construct a three-dimensional scene, and is used to identify the shape, position, distance, etc. of objects, providing visual information for the operation of the end effector. For example, it can assist in judging the position and posture of an object during a grasping task.

[0120] The first temperature sensor 6: It is used to monitor the temperature during the additive manufacturing process, such as the temperature of the first print head 10, the temperature field history of the manufacturing structure, etc., to ensure that the manufacturing process is carried out within a suitable temperature range, guarantee the material forming quality, and avoid manufacturing defects caused by abnormal temperature.

[0121] Manufacturing-related components: The first material supply device 5: It is responsible for providing the required materials for additive manufacturing, transporting the materials to the first print head 10, and is the source of material supply for realizing additive manufacturing.

[0122] The first print head 10: It deposits materials layer by layer according to instructions to realize the additive manufacturing process of the self-growing end effector and construct the target structure.

[0123] The first additive manufacturing substrate 9 and the second additive manufacturing substrate 11: As the basic platforms for additive manufacturing, they provide a supporting surface for material deposition, carry the printed and formed objects, and ensure the stability of the manufacturing process.

[0124] Basic support components: The first actuator base 4: It is the basic support structure of the entire end effector, providing an installation base and support for other components, ensuring the relative positions of the components are stable, and maintaining the stability of the overall structure.

[0125] Such as Figure 4As shown, it presents Case 2 of the end effector structure in a simulation environment. Compared with Case 1, the motion and control components are replaced by the first servo 22, the second servo 24, and the third servo 210: As power driving elements, they can precisely control the rotation angles and speeds of the first rotating joint 23 and the second rotating joint 25. By receiving control signals, they provide power support for the motion of the end effector, enabling precise attitude adjustment and position movement, and are key components for achieving flexible operation.

[0126] The first rotating joint 23 and the second rotating joint 25: In cooperation with the first servo 22, the second servo 24, and the third servo 210, they enable the corresponding components of the end effector to perform rotational motion around the axis, increasing the degrees of freedom of the end effector's motion in space. They can adjust the directions of components such as the second print head 212 and the second depth camera 26 to adapt to different operation requirements.

[0127] The explanations of the second depth camera 26, the second temperature sensor 29, the second print head 212, the third additive manufacturing substrate 21, the fourth additive manufacturing substrate 211, the second material supply device 28, and the second actuator base 27 can refer to the above Figure 3 explanation and will not be elaborated here.

[0128] As Figure 5 shown, it presents Case 3 of the end effector structure in a simulation environment. The mobile chassis 31: provides the entire system with the ability to move, enabling it to move flexibly in the working environment, allowing the end effector to reach different working positions, expanding the operation range, and being suitable for scenarios where tasks need to be performed in a relatively large space.

[0129] The robot 32: has multiple joints and degrees of freedom and can perform complex motion operations. It can precisely control the position and attitude of the end effector and accurately execute preset actions such as grasping, placing, and operating tools in additive manufacturing or other interaction tasks.

[0130] The additive manufacturing module 33: is the core part for realizing the additive manufacturing of the self-growing end effector. By the method of layer-by-layer material stacking, it constructs the target structure according to the design model, and can manufacture end effector components that meet different task requirements, providing a hardware foundation for the realization of the end effector's functions.

[0131] The multi-process tool library 34: stores tools of various different processes and can quickly replace tools according to task requirements. For example, during additive manufacturing, different types of print heads or processing tools can be switched as needed, and in other tasks, it can also be replaced with an adapted operation tool, enhancing the versatility and task adaptability of the end effector.

[0132] End effector 35 during the manufacturing process: It is gradually formed under the action of the additive manufacturing module and is a key component for finally performing specific tasks. Its structure and function are customized according to task requirements. After manufacturing, combined with the motion control of the robot, it can perform various target interaction tasks such as assembly, handling, and inspection.

[0133] As Figure 6 shown, it shows a schematic diagram of three interaction strategies of end effector structure case 1 for a sphere. Among them, Figure 6 (a) in it is a schematic diagram of sphere grasping; Figure 6 (b) in it is a schematic diagram of sphere adsorption; Figure 6 (c) in it is a schematic diagram of sphere lifting.

[0134] As Figure 7 shown, it shows two installation and application methods of end effector structure case 1: Figure 7 (a) in it presents the installation and application of a collaborative robotic arm. The self-growing end effector is installed on the collaborative robotic arm, and the collaborative robotic arm has multiple joints that can rotate flexibly.

[0135] Figure 7 (b) in it shows the installation and application of a drone. The self-growing end effector is carried under the drone, and the drone provides flight power through multiple rotors and has the ability to fly and hover flexibly in the air.

[0136] As Figure 8 shown, an interaction application method of end effector structure case 3, that is, the installation of a collaborative robotic arm and the sphere grasping application.

[0137] Figure 9 The schematic diagram of the structure of the end effector execution device provided by another embodiment of the present application is shown. For the sake of convenience of description, only the parts related to the embodiment of the present application are shown.

[0138] Referring to Figure 9 , the end effector execution device may include: A construction module 901, configured to construct a world model according to the environmental information of the end effector performing the target interaction task; the world model is used to characterize the physical characteristics and dynamic characteristics of the interaction environment of the end effector performing the target interaction task; A first determination module 902, configured to determine the target structure of the end effector and the interaction behavior strategy of the end effector performing the target interaction task based on a first preset model, according to the world model and the task requirements of the end effector; the first preset model is trained with the goal of maximizing the execution success rate of the target interaction task; The second determination module 903 is configured to determine, based on a second preset model, a structure manufacturing instruction for the end effector to manufacture the target structure and a quality regulation rule for the end effector to manufacture the target structure according to the target structure; and determine an interactive motion control instruction for the end effector to execute the interactive behavior strategy and a feedback compensation mechanism for the end effector to execute the target interactive task according to the interactive behavior strategy; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interactive robustness of executing the target interactive task. The first control module 904 is configured to control the end effector to manufacture the target structure in real time on demand through a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instruction, and the quality regulation rule. The second control module 905 is configured to control the target actuator to execute the target interactive task according to the interactive behavior strategy, the interactive motion control instruction, and the feedback compensation mechanism when the manufacturing of the target structure is completed, where the target actuator is the end effector with the target structure.

[0139] It should be noted that the information interaction, execution process, etc. between the above-mentioned device / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the above-mentioned method. All implementation manners in the method embodiment are applicable to the embodiment of this device. For its specific functions and the technical effects brought, please refer to the method embodiment section, which will not be elaborated here.

[0140] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be elaborated here.

[0141] Figure 10 The hardware structure diagram of the electronic device provided by the embodiment of the present application is shown.

[0142] The device may include a processor 1001 and a memory 1002 storing program instructions.

[0143] When the processor 1001 executes the program, it implements the steps in any of the above method embodiments.

[0144] Exemplarily, the program may be divided into one or more modules / units, and one or more modules / units are stored in the memory 1002 and executed by the processor 1001 to complete this application. One or more modules / units may be a series of program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the program in the device.

[0145] Specifically, the above-mentioned processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits implementing the embodiments of this application.

[0146] The memory 1002 may include a mass storage for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 1002 may include removable or non-removable (or fixed) media. In a suitable case, the memory 1002 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1002 is a non-volatile solid state memory.

[0147] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present disclosure.

[0148] The processor 1001 reads and executes the program instructions stored in the memory 1002 to implement any of the above methods.

[0149] In one example, the electronic device may further include a communication interface 1003 and a bus 1010. Among them, the processor 1001, the memory 1002, and the communication interface 1003 are connected through the bus 1010 to complete communication with each other.

[0150] The communication interface 1003 is mainly used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application.

[0151] The bus 1010 includes hardware, software, or both, and couples the components of the online data flow metering device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, the bus 1010 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0152] In addition, in combination with the method in the above embodiments, the embodiments of the present application may provide a storage medium to implement. Program instructions are stored on the storage medium; when the program instructions are executed by the processor, any one of the methods in the above embodiments is implemented.

[0153] The embodiments of the present application further provide a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0154] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system chip, a chip system, or a system-on-a-chip, etc.

[0155] The embodiments of the present application provide a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement each process of the above method embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0156] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0157] The functional modules shown in the above structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet or an intranet.

[0158] It should also be noted that in the exemplary embodiments mentioned in the present application, some methods or systems are described based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0159] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each block in the block diagrams and / or flowcharts, and the combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0160] The above are only specific implementation manners of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered within the protection scope of the present application.

Claims

1. An end effector execution method, characterized in that, The method includes: Constructing a world model according to the environmental information of the end effector performing the target interaction task; the world model is used to characterize the physical and dynamic characteristics of the interaction environment in which the end effector performs the target interaction task; Based on a first preset model, determining the target structure of the end effector and the interaction behavior strategy for the end effector to perform the target interaction task according to the world model and the task requirements of the end effector; the first preset model is trained with the goal of maximizing the execution success rate of the target interaction task; Based on a second preset model, determining the structural manufacturing instruction for the end effector to manufacture the target structure and the quality control rule for the end effector to manufacture the target structure according to the target structure; and determining the interaction motion control instruction for the end effector to execute the interaction behavior strategy and the feedback compensation mechanism for the end effector to execute the target interaction task according to the interaction behavior strategy; the second preset model is trained with the goal of maximizing the structural quality of the target structure and the interaction robustness of executing the target interaction task; Controlling the end effector to manufacture the target structure in real time on demand through multi-process additive manufacturing function according to the world model, the target structure, the structural manufacturing instruction and the quality control rule; When the manufacturing of the target structure is completed, controlling the target actuator to perform the target interaction task according to the interaction behavior strategy, the interaction motion control instruction and the feedback compensation mechanism, where the target actuator is the end effector with the target structure.

2. The method according to claim 1, wherein The constructing a world model according to the environmental information of the end effector performing the target interaction task includes: Obtaining the environmental information, where the environmental information includes the interaction environment, the interaction object of the interaction task, and the geometric and physical information of the actuator body of the end effector; Performing time calibration and feature extraction on the environmental information to obtain target feature information; Constructing the world model according to the target feature information, where the world model is used to realize the dynamic perception and prediction of the interaction environment.

3. The method according to claim 2, wherein The performing time calibration and feature extraction on the environmental information to obtain target feature information includes: Establishing a unified coordinate system for the environmental information collected from different acquisition channels based on a three-dimensional space registration algorithm to obtain first information; Performing time calibration on the first information through a distributed timestamp combined with a least squares algorithm to obtain calibrated information; Performing feature extraction on the calibrated information to obtain the target feature information.

4. The method according to claim 3, wherein The performing feature extraction on the calibrated information to obtain the target feature information includes: When the calibrated information is geometric point cloud data, performing feature extraction on the geometric point cloud data through a principal component analysis method to obtain the eigenvalues corresponding to the geometric point cloud data; When the calibrated information is the motion pose sequence of the end effector, extracting the joint friction coefficient of the motion pose sequence. When the calibration information is temperature field data, the core time-frequency information of temperature fluctuations in the temperature field data is extracted through Fourier transform; Among them, the target feature information includes the eigenvalue, the joint friction coefficient, and the core time-frequency information.

5. The method according to claim 2, characterized in that, The constructing the world model according to the target feature information includes: Reconstructing a three-dimensional geometric model of the interaction environment through implicit neural representation according to the target feature information corresponding to the interaction environment; Constructing a multi-physical property model of the interaction object through a material constitutive equation according to the target feature information corresponding to the interaction object; Determining the body dynamics model of the actuator body through a Lie group motion chain according to the target feature information corresponding to the actuator body; Determining the interaction dynamics model of the actuator body and the interaction environment through Hamilton's equation according to the body dynamics model; Constructing an energy potential contact model describing the contact between the actuator body and the interaction object according to the target feature information; Constructing a state transition equation for predicting the multi-physical field evolution of the interaction environment according to the target feature information; Constructing the world model according to the three-dimensional geometric model, the multi-physical property model, the body dynamics model, the interaction dynamics model, the energy potential contact model, and the state transition equation.

6. The method according to claim 1, wherein The first preset model is trained with the goal of maximizing the execution success rate of the target interaction task, and includes: Constructing a first initial model that takes into account the environmental constraints of the interaction environment and the diversity of task requirements of the interaction task; Training the first initial model with a first preset training sample with the goal of maximizing the task execution success rate until a first preset training stop condition is met to obtain the first preset model; the first preset model is used to determine the target structure and the interaction behavior strategy, and the structural features of the target structure include the kinematic reachability, mechanism stiffness and compliance of the target structure; the interaction behavior strategy includes a basic action sequence and a safety policy library; The second preset model is trained with the goal of maximizing the structural quality of the target structure and the interaction robustness of executing the target interaction task, and includes: Constructing a second initial model; Training the second initial model with a second preset training sample with the goal of maximizing the structural quality of the target structure and the interaction robustness of executing the target interaction task until a second preset training stop condition is met to obtain the second preset model; among them, when the second preset model determines the structure manufacturing instruction and the quality control rule, path continuity and obstacle avoidance are considered; when the second preset model determines the interaction motion control instruction and the feedback compensation mechanism, different failure modes of the end effector are considered.

7. The method according to claim 1, characterized in that, The controlling the end effector to manufacture the target structure in real time on demand through a multi-process additive manufacturing function according to the world model, the target structure, the structure manufacturing instruction, and the quality control rule includes: During the process of manufacturing the target structure by the end effector according to the structure manufacturing instruction, based on the world model, the structure manufacturing instruction is adjusted according to the manufacturing state of the end effector at the current moment and the quality regulation rule, so that the manufacturing quality of the target structure meets the first preset quality.

8. The method according to claim 1, characterized in that Controlling the target actuator to execute the target interaction task according to the interaction behavior strategy, the interaction motion control instruction and the feedback compensation mechanism includes: During the process of the target actuator executing the target interaction task according to the interaction motion control instruction, based on the world model, the interaction motion control instruction is adjusted according to the interaction state of the target actuator at the current moment and the feedback compensation mechanism, so that the execution quality of the target actuator executing the target interaction task meets the second preset quality.

9. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the method for the end effector execution as described in any one of claims 1-8 is implemented.

10. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by the processor, the method for the end effector execution as described in any one of claims 1-8 is implemented.

Images