Electric power operation robot body intelligence model optimization method and system
By working together with the target detection stream and the embodied intelligent model visual encoding stream, and combining Gaussian attention graphs and hierarchical weighting mechanisms, the problem of insufficient target recognition and operation accuracy of general models in power operations is solved, and high-precision and stable power operation tasks are achieved.
Patent Information
- Application Number
- CN202511339747.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-19
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2045-09-19
AI Technical Summary
Existing general-purpose embodied intelligence models are easily affected by background interference in power operations, making it difficult to focus on key targets and resulting in a low success rate for precision operation tasks. In particular, the performance of identifying and tracking targets such as insulators deteriorates in complex environments.
The target detection stream and the embodied intelligence model visual encoding stream work together. By constructing a Gaussian attention map and a hierarchical weighting mechanism, the model is guided to focus on key targets and optimize action decisions.
This improved the target recognition accuracy and operational robustness of robots in power operations, achieving millimeter-level precision operation, increasing task success rate, and reducing upgrade costs.
Smart Images

Figure CN120816509B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of artificial intelligence, robotics, and power system automation, and specifically relates to a method and system for optimizing the embodied intelligent model of a power operation robot. Background Technology
[0002] In recent years, with the development of robotics and artificial intelligence, using live-line working robots to replace manual labor in high-altitude, high-voltage power distribution network environments has become an important direction for ensuring the safety of maintenance personnel and improving the efficiency of power grid operation and maintenance. Embodied AI models, especially large models that combine vision, language, and action (VLA), such as Google's RT-2, have demonstrated enormous potential to enable robots to understand and execute complex instructions. These models, pre-trained on massive amounts of internet data, can generalize to a variety of task scenarios.
[0003] For example, patent application CN120228719A employs multimodal perception fusion technology to generate intelligent decisions based on a large model, and executes nuclear industry operations through a robotic arm and force feedback control system. This embodied intelligent design enables the robot to autonomously adapt to the complex environment of the nuclear industry, improving operational efficiency and safety. Patent application CN120382487A proposes an embodied system that uses a large model-driven task parsing and decomposition technology to transform complex instructions into executable steps. It also employs multimodal perception fusion technology to plan operational feedback and a force feedback mechanism to ensure execution accuracy, achieving human-robot collaborative operation and improving the robot's task execution capability and adaptability in complex environments.
[0004] However, when applied to scenarios with high professionalism and safety requirements, such as power operations, especially power operations with high precision requirements (e.g., insulator maintenance), existing general VLA models face the following challenges: (1) they are easily affected by background interference in complex operation scenarios, which leads to distraction and difficulty in focusing on key operation targets; (2) they are difficult to meet the requirements of high precision and stability in fine operations such as grasping, alignment, and pin insertion at the millimeter level; (3) they lack robustness in non-ideal environments such as lighting, weather, or shading, which reduces the recognition and tracking performance of targets such as insulators. Summary of the Invention
[0005] This invention aims to address the problems of existing general-purpose embodied intelligent models when applied to specialized scenarios such as power operations, including easy distraction, inaccurate perception of key targets, and low success rate in delicate operational tasks. Specifically, this invention strives to provide a method that can significantly improve the target positioning accuracy, operational robustness, and final task success rate of embodied intelligent models in tasks such as insulator replacement.
[0006] The first aspect of this application provides a method for optimizing the embodied intelligent model of a power operation robot, which adopts the following technical solution:
[0007] Images of insulators in the current work scenario are acquired and input into the target detection stream and the embodied intelligent model visual encoding stream, respectively. The target detection stream outputs a list of bounding boxes of target insulators, and the embodied intelligent model visual encoding stream outputs a global feature map.
[0008] Select the bounding box with the highest confidence value from the bounding box list, transform its center point coordinates from the image coordinate system to the global feature map coordinate system, and generate an index position; construct a two-dimensional Gaussian distribution with the two coordinates of the index position as the center point, and construct a Gaussian attention map.
[0009] The global feature map and the Gaussian attention map are multiplied element-wise to obtain the modulation feature map; the modulation feature map is input into the embodied intelligence model, and combined with the task instructions, a sequence of instructions is generated to control the execution of the robot arm.
[0010] Furthermore, the target detection stream employs a 2D target detection model, which is trained using a power scene dataset;
[0011] The insulator image is input into the trained 2D target detection model, which outputs a list of bounding boxes for the target insulators; each bounding box contains information. ,in These are the pixel coordinates of the center point of the bounding box. and These are the width and height of the bounding box, respectively. The confidence level of the bounding box;
[0012] Set a confidence threshold for the bounding box and filter out bounding boxes with a confidence level greater than or equal to the confidence threshold. A bounding box.
[0013] Furthermore, the processing of the embodied intelligent model visual encoding stream includes:
[0014] The insulator image is segmented into a grid with an equal number of rows and columns using a visual encoder, with each grid cell being an image block; the visual encoder generates an initial feature vector for each image block;
[0015] For the A bounding box is defined, and its center point is mapped from image coordinates to image patch coordinates to obtain the target center index.
[0016] Calculate the first The center point of the image patch is related to the first The distance to the target center index is used to generate the first target center index. Image patches in the bounding box The weights below; take all The maximum value among the weights is used as the weight of the first weight. The weighted weights of each image patch;
[0017] The initial feature vector of each image patch is multiplied by its corresponding weight to obtain a weighted feature vector. All weighted feature vectors are combined to output a global feature map.
[0018] Furthermore, the first Image patches in the bounding box The weights below are the sum of the exponential decay term and the constant offset term;
[0019] The exponential decay term is the product of the weighted intensity control parameter and the standard normal distribution term, in which the mean is replaced by the distance and the standard deviation is replaced by the attention decay rate control parameter.
[0020] The constant offset term is the difference between 1 and the weighted intensity control parameter.
[0021] Furthermore, in Select the bounding box corresponding to the highest confidence score from the given bounding boxes. The center pixel Transform from the image coordinate system to the global feature map coordinate system; obtain the index position of the maximum target center on the global feature map. ;
[0022] Index coordinates The method of obtaining it is pixel coordinates. The ratio of the insulator image height to the grid column number, multiplied by the grid column number, with the product rounded down; index coordinates. The method of obtaining is pixel coordinates The ratio of the width of the insulator image to the number of grid rows is multiplied by the number of grid rows, and the product is rounded down.
[0023] Furthermore, using index coordinates and index coordinates As two means, a two-dimensional Gaussian distribution is constructed to generate the Gaussian attention map, the dimension of which is the same as the dimension of the global feature map;
[0024] And for each location in the Gaussian attention map Calculate the Gaussian attention value; the Gaussian attention value is the result of operating on a two-dimensional Gaussian distribution function, where the numerator of the first fraction in the exponent is the Gaussian coordinate. with index coordinates The difference is squared, and the denominator is twice the square of the first standard deviation; the numerator of the second fraction in the exponent is a Gaussian coordinate. with index coordinates The square of the difference is 2 times the square of the second standard deviation.
[0025] Furthermore, the first standard deviation is the product of the adjustable scaling hyperparameter and the bounding box height percentage, where the bounding box height percentage refers to... The ratio of the height of the insulator to the height of the insulator image multiplied by the number of grid columns;
[0026] The second standard deviation is the product of the adjustable scaling hyperparameter and the bounding box width percentage, where the bounding box width percentage refers to... The ratio of the width of the image to the width of the insulator image is multiplied by the number of grid rows.
[0027] A second aspect of this application provides an embodied intelligent model optimization system for power operation robots, employing the embodied intelligent model optimization method as described in the first aspect of this application, the system comprising:
[0028] A dual-channel processing module is used to acquire images of insulators in the current working scenario and input them into a target detection stream and an embodied intelligent model visual encoding stream, respectively. The target detection stream outputs a list of bounding boxes of target insulators, and the embodied intelligent model visual encoding stream outputs a global feature map.
[0029] The adaptive focusing module is used to select the bounding box corresponding to the maximum confidence value in the bounding box list, transform the center point coordinates of the bounding box from the image coordinate system to the global feature map coordinate system, and generate an index position; and construct a two-dimensional Gaussian distribution with the two coordinate values of the index position as the center point to construct a Gaussian attention map.
[0030] An embodied intelligent model execution module is used to perform element-wise multiplication of the global feature map and the Gaussian attention map to obtain a modulation feature map; the modulation feature map is input into the embodied intelligent model, and combined with task instructions, an instruction sequence for controlling the execution of the robot arm is generated. The present invention adopts the following technical solution.
[0031] Compared with the prior art, the technical solution proposed in this application has at least one of the following beneficial effects:
[0032] 1. This application employs a collaborative approach between a target detection stream and a VLA (Visual Algorithm) encoding stream, enabling the robot to simultaneously acquire the precise location of the insulator and global scene features. During the "grabbing the insulator" action, the target detection stream provides bounding box center information, allowing the robotic arm to quickly locate the insulator. During the "aligning the pin hole" and "inserting the pin" actions, the visual encoding stream provides global spatial features, allowing the robot to consider surrounding interference elements such as utility poles, crossarms, and conductors, achieving continuous and stable precision operations. This bidirectional processing stream design enhances the robot's target recognition accuracy and motion consistency in complex power operation scenarios.
[0033] 2. This application proposes a hierarchical weighting mechanism that enables the robot to progressively focus on key target regions in the feature map. Initial weighting of image blocks is applied during the visual encoding stream, and further modulation is achieved using a Gaussian attention map during feature map fusion. When performing the "grabbing insulator" action, the initial weighting prioritizes the VLA's focus on the insulator center. During the "aligning pin holes" and "inserting pins" actions, the target region features are further enhanced while background interference is reduced, enabling the robot to achieve millimeter-level precision. This hierarchical weighting mechanism improves the robot's operational accuracy and task success rate under complex backgrounds and varying distances.
[0034] 3. The technical solution proposed in this application can be embedded as a "plug-and-play" attention enhancement module into various VLA models based on the ViT architecture without requiring major structural changes or complete retraining of the main VLA model, thus reducing the cost of technology application and upgrades. The method proposed in this application also possesses good task scalability. By changing the training data of the target detection model and the detection targets (such as bolts, crimped pipes, wires, etc.), this method can be easily transferred and applied to other power operation tasks, demonstrating strong versatility and expansion potential. Attached Figure Description
[0035] Figure 1 This is a flowchart illustrating the embodied intelligence model optimization method provided in this embodiment. Detailed Implementation
[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.
[0037] The background of power operation scenarios is complex, with numerous elements such as power poles, crossarms, conductors, fittings, and distant buildings and trees. When processing visual input, general-purpose VLA models may fail to accurately focus attention on the most critical object of the current task (such as the insulator to be replaced or a specific part of it), and are easily affected by background interference, leading to biased action decisions. The task of replacing an insulator involves a series of delicate operations such as "grabbing the insulator," "aligning the pin hole," and "inserting the pin." Due to its generalization characteristics, general-purpose VLA models often struggle to achieve the required accuracy and stability for such millimeter-level positioning and alignment tasks. Although VLA models have some object recognition capabilities, their ability to identify and continuously track power components like insulators, which are simple in shape and similar in color, decreases under non-ideal conditions such as complex lighting, weather changes, and partial occlusion, affecting the continuity and success rate of the task. Therefore, how to guide general-purpose embodied intelligence models to accurately "focus on" and "operate" specific task targets in complex environments, like domain experts, is a key technical bottleneck for their successful application in live-line power operations.
[0038] To address the aforementioned technical problems, this application provides specific technical implementation steps for an embodied intelligent model optimization method for power operation robots in the detailed implementation section. The core idea of this method is to utilize an efficient and accurate 2D target detection model (first "seeing" the key target), and integrate its output positional prior information into the visual feature processing flow of a general embodied intelligent model (VLA model). By constructing a target-centered attention "focus," the VLA model is forced to focus on the key object, thereby optimizing its subsequent action decisions.
[0039] In a typical application scenario of replacing insulators with a live-line working robot in a power distribution network, as an embodiment of this application, the technical steps of an embodied intelligent model optimization method for power work robots are given.
[0040] As one implementation step of this embodiment, the live-line working system is initialized, including a robot platform, a target detection model, and an embodied intelligence model. The robot is a live-line working robot equipped with a six-axis collaborative robotic arm and a wrist-mounted RGB camera. The camera resolution is 1920x1080. The target detection model uses the YOLOv8 model, fine-tuned on a dataset containing 100,000 images of power scene scenarios. The dataset covers different types of pin insulators and suspension insulator strings, as well as samples under different lighting and weather conditions such as sunny, cloudy, rainy, and dusk. The embodied intelligence model uses an open-source VLA model based on Transformer as its base model. Its visual encoder divides the input image into 16x16 image patches, i.e., N=16.
[0041] S1: Collect real-time images of the current work scene and obtain input data;
[0042] After the live-line working robot moves to the designated work point, the operator issues the command "Start replacing phase 2 insulator". Upon receiving the command, the robot uses a vision sensor (such as an RGB camera) mounted on its wrist to capture a 1920×1080 pixel image of the current work scene. This image will serve as a unified input source for subsequent processing.
[0043] As one implementation method of this embodiment, the exposure parameters of the wrist vision sensor are adjusted according to the ambient light intensity, reflection conditions, and background complexity to acquire multiple frames of images of the target insulator within one sampling period. These multiple frames are then fused to generate an image. .
[0044] S2: The acquired scene image The input consists of two processing streams: an object detection stream and an embodied intelligence model visual encoding stream.
[0045] 2.1 Object Detection Stream: This stream processes the image... Input a pre-trained 2D object detection model The target detection model has been fine-tuned using a power scene dataset (containing insulator images of various types, angles, and lighting conditions) and has the ability to detect insulators with high precision.
[0046] The model outputs a list of bounding boxes for the target insulators, with each bounding box containing information. ,in These are the pixel coordinates of the center point of the bounding box. and These are the width and height of the bounding box, respectively. This represents the confidence level of the bounding box.
[0047] Below the confidence threshold Bounding boxes are removed, retaining only those with a confidence level greater than or equal to [value missing]. The bounding box is used as input for subsequent feature fusion, providing accurate and reliable localization for the robot arm's "grasping action".
[0048] 2.2 Embodied Intelligent Model Visual Encoding Stream: Transforming images... After preprocessing according to the resolution required by ViT, it is input into the general embodied intelligence model. visual encoder This visual encoder uses a visual (Transformer) architecture to process images. Divided into The grid consists of a grid, and each grid cell is called a patch.
[0049] 2.2.1: Visual Encoder All patches are processed, and a feature vector is generated for each patch.
[0050] 2.2.2: Each patch uses its center coordinates This indicates that the multiple bounding boxes output by S2.1 are mapped to the Patch coordinate system.
[0051] For the A bounding box, with its center point Take its center point Mapping from image coordinates to the Patch coordinate system yields the target center index. ;
[0052] For the One patch, Calculate its distance to the target center. :
[0053] ;
[0054] According to distance Generate the first The weight of each patch The calculation method is as follows:
[0055] ;
[0056] in, The weighted intensity control parameter for Patch is used to control the strength of the influence of distance on the final weight value; This is a parameter for controlling the rate at which attention decays, used to control the rate at which the weight value decays with increasing distance.
[0057] The same method applies to the rest. Weights are calculated for each bounding box, and then... The maximum value among the weights is used as the weight of the first weight. The weight of each patch .
[0058] 2.2.3: Multiply each patch by its corresponding weight to obtain a weighted feature vector, and then combine all the weighted patch vectors to output a feature map. ,in This is the dimension of the feature vector for each patch. Weighted patches amplify the features of patches closer to the target center and weaken the features of patches farther from the target.
[0059] Embodied Intelligent Vision Encoding Stream (VLA) can extract global features, providing high-dimensional perceptual information for continuous actions such as "aligning pin holes" and "inserting pins," thereby improving the coherence and stability of insulator replacement operations. In conventional ViT, all patches are typically treated equally, with their feature vectors directly generated. However, to improve the accuracy of operation location positioning, this embodiment uses the distance to the bounding box to assign weights to different patches, allowing the VLA model to naturally focus on the insulator and key operation parts during the visual feature processing stage, reducing interference from the background.
[0060] By assigning higher weights to the target area patches, key objectives are focused, making subsequent robotic arm movements (grasping, aligning pin holes, inserting pins) more precise and improving the success rate of operations. Furthermore, weighting at the ViT patch layer requires no changes to the core structure of the ViT model, ensuring compatibility with existing VLA models and Transformer encoders, and making deployment convenient and easy to implement.
[0061] S3: Establish the spatial correspondence between bounding box information and ViT feature maps;
[0062] In the bounding box list output by the object detection stream, select the bounding box corresponding to the maximum confidence score and set its center pixel. Transform from the original image coordinate system to Using the feature map coordinate system, we obtain the index position of the maximum target center on the feature map. The conversion method is expressed as:
[0063] ;
[0064] in, and For image The original image height and width, This is the floor function. This is the "desired position," or "alignment position," that the VLA model should focus on.
[0065] S4: Construct an adaptive Gaussian attention graph.
[0066] by In and A two-dimensional Gaussian distribution is constructed using the means of the two dimensions, generating a... Gaussian attention map Each position on the map value The calculation is as follows:
[0067] ;
[0068] in, and The standard deviations of the Gaussian distribution across two dimensions determine the size of the attention "focus." The bounding box sizes corresponding to the two standard deviations and the maximum confidence score are also considered. and Positive correlation, for example, can be set as:
[0069] ;
[0070] in, It is an adjustable scaling hyperparameter. Thus, when the camera is far from the insulator (smaller detection box), the Gaussian distribution is sharper and the focus is more concentrated; when it is close to the insulator (larger detection box), the Gaussian distribution is flatter and the focus covers the entire target.
[0071] S5: Utilize the global feature map Gaussian attention map Element-wise multiplication yields the modulated new feature map. :
[0072] ;
[0073] This operation reduces the distance from the target center. The closer the image patch, the higher its feature vector weight, and vice versa. This is equivalent to putting a "scope" on the "visual cortex" of the VLA model, forcing it to focus on the key objects identified by the object detection model. This step ensures that the robot can identify targets more accurately when performing "grasping" and "aligning" actions, while providing the necessary high-precision feature support for millimeter-level operations such as "inserting pins".
[0074] As an embodiment of this application, the attention-modulated feature map Send into the general embodied intelligence model Subsequent modules (such as the Transformer decoder), combined with task instructions (such as "replace the insulator"), ultimately generate a sequence of instructions (such as joint angles, end-effector poses, etc.) to control the robot arm to perform corresponding actions.
[0075] As an illustration of this embodiment, in section 2.2, the feature vector of each patch is initially weighted, that is, the feature vector is assigned a weight based on the distance between the patch and the center of the target bounding box, so that the VLA model initially focuses on the key target region during the visual encoding stage.
[0076] Step 5, based on this, involves processing the preliminary weighted feature map. Further modulation is performed to enhance the features of the target region and suppress the background features. The standard deviation of the Gaussian attention can be adaptively adjusted according to the bounding box size, so that both near and far targets can achieve a reasonable focusing effect. Thus, steps 2.2 and 5 form a progressive attention guidance mechanism: step 2.2 provides the initial weight allocation principle, and step 5 realizes systematic, smooth, and adjustable high-precision feature modulation, thereby significantly improving the VLA model's recognition accuracy and action decision stability for key targets in complex power operation scenarios in a hierarchical manner.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.
Claims
1. A method for optimizing the embodied intelligent model of a power operation robot, characterized in that, include: Collect images of insulators in the current work scenario and input them into the target detection stream and the embodied intelligent model visual encoding stream, respectively. The target detection stream outputs a list of bounding boxes of target insulators, and the embodied intelligent model visual encoding stream outputs a global feature map. The target detection stream uses a 2D target detection model, which is trained using a dataset of power equipment images from the power operation robot's work scenario. The insulator image is input into the trained 2D target detection model, which outputs a list of bounding boxes for the target insulators; each bounding box contains information. ,in These are the pixel coordinates of the center point of the bounding box. and These are the width and height of the bounding box, respectively. The confidence level of the bounding box; Set a confidence threshold for the bounding box and filter out bounding boxes with a confidence level greater than or equal to the confidence threshold. One bounding box; The processing of the embodied intelligent model's visual encoding stream includes: The insulator image is segmented into a grid with an equal number of rows and columns using a visual encoder, with each grid cell being an image block; the visual encoder generates an initial feature vector for each image block; For the The center point of each bounding box is mapped from the image coordinate system to the image patch coordinate system to obtain the target center index. Calculate the first The center point of the image patch is related to the first Distance of each target center index , generate the first Image patches in the bounding box The weights below , represented as: ; in, Weighted intensity control parameters for image blocks; This is a parameter for controlling the rate of attention decay; Take all The maximum value among the weights is used as the weight of the first weight. The initial feature vector of each image patch is multiplied by its corresponding weight to obtain a weighted feature vector. All weighted feature vectors are combined to output a global feature map. Select the bounding box with the highest confidence value from the bounding box list, transform its center point coordinates from the image coordinate system to the coordinate system of the global feature map, and generate an index position; construct a two-dimensional Gaussian distribution with the two coordinate values of the index position as the center point, and construct a Gaussian attention map; The global feature map and the Gaussian attention map are multiplied element-wise to obtain the modulation feature map; the modulation feature map is input into the embodied intelligence model, and combined with the task instructions, a sequence of instructions is generated to control the execution of the robot arm.
2. The method for optimizing the embodied intelligent model of a power operation robot according to claim 1, characterized in that, exist Select the bounding box corresponding to the highest confidence score from the given bounding boxes. The center pixel Transform from the image coordinate system to the coordinate system of the global feature map; Obtain the index position of the maximum target center on the global feature map. ; Index coordinates The method of obtaining it is pixel coordinates. The ratio of the insulator image height to the grid column number, multiplied by the grid column number, with the product rounded down; index coordinates. The method of obtaining is pixel coordinates The ratio of the width of the insulator image to the number of grid rows is multiplied by the number of grid rows, and the product is rounded down.
3. The method for optimizing the embodied intelligent model of a power operation robot according to claim 2, characterized in that, index coordinates and index coordinates As two means, a two-dimensional Gaussian distribution is constructed to generate the Gaussian attention map, the dimension of which is the same as the dimension of the global feature map; And for each location in the Gaussian attention map Calculate the Gaussian attention value; the Gaussian attention value is the result of operating on a two-dimensional Gaussian distribution function, where the numerator of the first fraction in the exponent is the Gaussian coordinate. with index coordinates The difference is squared, and the denominator is twice the square of the first standard deviation; the numerator of the second fraction in the exponent is a Gaussian coordinate. with index coordinates The square of the difference is 2 times the square of the second standard deviation.
4. The method for optimizing the embodied intelligent model of a power operation robot according to claim 3, characterized in that, The first standard deviation is the product of the adjustable scaling hyperparameter and the bounding box height percentage, where the bounding box height percentage refers to... The ratio of the height of the insulator to the height of the insulator image multiplied by the number of grid columns; The second standard deviation is the product of the adjustable scaling hyperparameter and the bounding box width percentage, where the bounding box width percentage refers to... The ratio of the width of the image to the width of the insulator image is multiplied by the number of grid rows.
5. A system for optimizing the embodied intelligent model of a power operation robot, employing the embodied intelligent model optimization method as described in any one of claims 1-4, characterized in that, The system includes: A dual-channel processing module is used to acquire images of insulators in the current working scenario and input them into a target detection stream and an embodied intelligent model visual encoding stream, respectively. The target detection stream outputs a list of bounding boxes of target insulators, and the embodied intelligent model visual encoding stream outputs a global feature map. The adaptive focusing module is used to select the bounding box corresponding to the maximum confidence value in the bounding box list, transform the center point coordinates of the bounding box from the image coordinate system to the global feature map coordinate system, and generate an index position; and construct a two-dimensional Gaussian distribution with the two coordinate values of the index position as the center point to construct a Gaussian attention map. An embodied intelligent model execution module is used to perform element-wise multiplication of the global feature map and the Gaussian attention map to obtain a modulation feature map; input the modulation feature map into the embodied intelligent model, and combine it with task instructions to generate a sequence of instructions for controlling the execution of the robot arm.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the embodied intelligence model optimization method according to any one of claims 1-4.
7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the embodied intelligence model optimization method according to any one of claims 1-4.
Citation Information
Patent Citations
Nuclear industrial robot body intelligent system based on large model
CN120228719A
Large-model-based robot brain body-equipped intelligent system and task execution method thereof
CN120382487A
Adaptive image alignment using locally optimal transformations
CN110603569A
Insulator defect detection method and device, medium and equipment
CN120219919A