A body-aware intelligent visual language action model for hinge joint manipulation

Abstract

The application discloses a somatic intelligent visual language action model for hinge joint operation, which comprises an image text multi-modal feature matching module, a visual language model, an object key point prediction and gripper pose prediction module, an object motion trajectory prediction module and an adaptive strategy selection module. The model is fused with a visual-language-action architecture, image text multi-modal technology, a visual language model and a group relative strategy optimization method, realizes the identification of home object operation key points, the estimation of mechanical arm poses and the prediction of motion trajectories, and thus completes the accurate operation of objects. The application selects three types of high-frequency interactive objects, i.e. washing machines, refrigerator doors and drawers, for testing. The washing machine recognition accuracy is improved from 78% of a traditional algorithm to 96%; the refrigerator door recognition accuracy is improved to 76%; and the drawer recognition accuracy is improved from 71% to 78%, which greatly improves the understanding and operation precision of the somatic housekeeper for hinge objects.

Description

Technical Field

[0001] This invention relates to an embodied intelligent visual language action model, specifically an embodied intelligent visual language action model for hinge joint control. Background Technology

[0002] In the current era of rapid development in intelligent manufacturing and service robot technology, the ability of robots to autonomously complete human-robot collaborative tasks in complex environments has become a core indicator for measuring the performance of intelligent agents. From equipment assembly on industrial production lines and goods storage and retrieval in warehousing and logistics, to home control and elderly care in home scenarios, robots need to accurately understand human language commands, recognize visual environmental information, and execute compliant physical actions. This places extremely high demands on multimodal perception and decision-making capabilities. The Visual-Language-Motion Model (VLAM), as an advanced multimodal intelligent model built on deep neural networks, breaks through the limitations of single-modal technologies by leveraging its core advantage of collaboratively processing visual images, natural language text, and physical action commands. It has become a universal and cutting-edge technical means to solve the closed-loop problem of cross-modal perception-decision-action in robots.

[0003] In human daily work and living environments, articulated objects (such as doors, drawers, cabinets, refrigerators, and laptops) constitute a large proportion. These objects are not only numerous but also form key nodes for functional interaction in physical space. Articulated objects are a fundamental type of object in human production and daily life, with applications covering homes, industry, commerce, and warehousing, and are closely related to daily living and production operations. Therefore, achieving autonomous operation of articulated objects is the foundation and prerequisite for robots to achieve service-oriented and intelligent functions. Their importance lies not only in their wide application but also in the complexity of interaction brought about by their structural characteristics: unlike ordinary rigid objects such as water cups and books, they have fixed structures such as rotation axes and sliding rails, requiring specific actions such as rotation around the axis and sliding along the rail to achieve state changes, and these actions must adhere to physical constraints.

[0004] Compared to simple, rigid objects like water cups and books, the autonomous operation of hinged objects presents an exponentially greater challenge, with the core technological hurdle stemming from their unique structural characteristics and interaction logic. On one hand, hinged objects possess clearly defined rotation axes, sliding rails, and other fixed constraints. Changes in their state must follow specific physical trajectories; for example, cabinet doors must rotate around their hinge axes, and drawers must slide linearly along guide rails. Any deviation from these constraints will result in operational failure. On the other hand, the interaction targets of hinged objects are often concealed. Their core functional areas (such as the interior of drawers or storage spaces behind cabinet doors) often require unlocking through compliant actions to reach. The robot must simultaneously complete a coherent process of "state recognition - action planning - physical execution," which places stringent demands on the accuracy of multimodal information fusion and the rationality of action decisions.

[0005] Although VLAM has cross-modal semantic alignment and action mapping capabilities through pre-training on massive amounts of "image-text-action" triplet data, enabling end-to-end execution of "perception-understanding-decision-action", existing VLAM-based technical solutions still have significant limitations when adapting to hinged object manipulation tasks, making it difficult to fully meet the application needs of real-world complex scenarios. Firstly, in the grasping point localization stage, existing models mostly rely on general key point detection logic, failing to design specific feature extraction and localization strategies for the structural characteristics of hinged objects. This makes it difficult to accurately identify the optimal grasping area, easily leading to "grasping offset" issues, such as mistakenly grasping the edge of a cabinet door instead of the handle, or grasping the drawer panel instead of the pull handle, directly resulting in ineffective force application in subsequent actions. Secondly, in the end effector adaptation stage, most methods default to using suction cups as the core actuator, ignoring the common characteristics of hinged objects such as "small contact area" and "complex surface materials." In small contact areas such as cabinet door handles and drawer pulls, suction cups are difficult to form effective negative pressure fixation, making it more suitable to achieve precise gripping with grippers. However, existing models lack dynamic selection and adaptation logic for actuator types, resulting in a serious disconnect from the manipulation needs of real industrial or home scenarios. Furthermore, the performance improvement of VLAM is also constrained by inherent problems such as high cost of motion data annotation and difficulty in multimodal temporal synchronization, further restricting its practical application in hinged object manipulation scenarios.

[0006] ManipLLM achieves stable and generalized robot manipulation using a multimodal large language model. It receives text and image (color + depth) information and outputs text containing two vectors (upward and forward) from the robotic arm's end effector gripper. The robotic arm then manipulates objects based on this information. In terms of model structure, the color image is encoded using CLIP's visual encoder, and the prompt text is processed by a pre-trained LLaMa word segmenter to obtain text features. These features are then fused and aligned through a projection module before being input into LLaMa, which performs multimodal (text and image) understanding and provides corresponding responses. During training, only the visual adapter, projection module, and language adapter are fine-tuned, while other parameters remain unchanged, aiming to preserve the capabilities of the original large model as much as possible. The model fine-tuning process employs four different prompts for training, each with distinct data and tasks: The object category recognition task prompts "What category does the object in the image belong to?", utilizing the existing large model's understanding to classify objects without updating model parameters. The functional prior reasoning task primarily requires the large model to determine the manipulable parts of an object, with the prompt "Determine whether manipulating the following points can effectively manipulate the object in the image: (x1, y1)...(x2, y2)", essentially asking the large language model to judge whether given points can be used to manipulate the object. This step fine-tunes the model based on positive and negative samples from the functional graph, enabling the large model to determine which pixels can be used to manipulate the object. The functional graph treats objects as geometric shapes, indicating the probability of movement distance when manipulating certain pixels, reflecting the position of the manipulable object. The fine-tuning and masking language modeling task primarily requires the large model to generate accurate end-effector operation posture positions, with the prompt "Specify the contact point of the manipulated object and the gripper direction". Relevant data is collected in the simulation environment; if the operation is successful, the corresponding color image and end-effector posture are recorded. A masking language modeling approach is used. After masking the vectors, the larger model fills in the masked positions. After fine-tuning, a thought process is used during actual operation to guide the larger model step by step to output the correct operation vectors.

[0007] The above method has the following problems:

[0008] (1) When the predictive model outputs the operation point, it selects random points on the movable operation surface of the hinge object. These points often cannot match the gripping shape and force angle of the gripper, which is seriously inconsistent with the precise operation requirements of the robotic arm on the hinge object (such as hinge cabinet door, rotary buckle) in the real scene.

[0009] (2) The model has poor generalization ability. When recognizing and manipulating hinge objects, it can only adapt to rotating hinges (such as cabinet door hinges and door and window pivots) seen in the training set. When faced with translational hinges (such as drawer slides and sliding door tracks) that did not appear in the training, the model will have problems such as incorrect object type recognition and complete failure of operation strategy.

[0010] (3) During the process of manipulating the hinged object by the model control robot arm, it is impossible to continuously track the movement trajectory of the movable part, which makes it very easy for the robot arm to jam or even collide with the fixed base of the hinge when rotating the hinge moving part, affecting the stability and safety of the operation. Summary of the Invention

[0011] This invention provides an embodied intelligent visual language action model for hinge joint manipulation. This model integrates a visual-language-action architecture, image-text multimodal technology, visual language model, and group relative strategy optimization to achieve the identification of key points for manipulating home objects, estimation of robotic arm pose, and prediction of motion trajectory, thereby enabling precise manipulation of objects.

[0012] The objective of this invention is achieved through the following technical solution:

[0013] An embodied intelligent visual language action model for hinge joint manipulation includes an image-text multimodal feature matching module, a visual language model (VLM), an object keypoint prediction and gripper pose prediction module, an object motion trajectory prediction module, and an adaptive strategy selection module, wherein:

[0014] The image-text multimodal feature matching module is responsible for aligning input information from both text and image modalities, responding to home operation needs, and completing the fusion and construction of multimodal commands by encoding information from different modalities into a unified vector representation space.

[0015] The visual language model is responsible for receiving multimodal instruction segments and performing high-level semantic understanding and decision generation, outputting accurate predictions of key points of robotic arm operation, including gripping position, axis direction, end effector pose and motion trajectory.

[0016] The object key point prediction and gripper pose prediction module is responsible for using the geometric and physical characteristics of the home hinge object as a basis, clarifying the core operation parameters through modeling, and then combining the algorithm design to output key information, and finally deriving the appropriate pose constraint reward function to provide decision support for the operation of the robotic arm.

[0017] The object motion trajectory prediction module is responsible for combining the motion characteristics and operational needs of the home objects, integrating information from multiple dimensions to output accurate trajectories, and providing support for the robotic arm to plan its operation path and avoid motion interference.

[0018] The adaptive policy selection module is responsible for adaptively selecting appropriate policies based on the different processes of the two core tasks of key point pose recognition and object trajectory prediction. It embeds the GRPO (Group Relative Policy Optimization) reinforcement learning optimization method into the training process of the visual language model, and improves the stability and convergence efficiency of policy training through the group relative policy evaluation mechanism.

[0019] A method for hinge joint manipulation using the aforementioned embodied intelligent visual language action model includes the following steps:

[0020] Step 1: Image-text multimodal feature matching:

[0021] Step 11: Assume the input data contains a text modality. With image modality Then the entire input set is represented as: For information from different modalities, separate feature encoders are designed: ,in, and Let represent the feature extraction functions for the text modality and the image modality, respectively, and let represent their feature vectors. It is the dimension of the text feature vector. It is the dimension of the visual feature vector;

[0022] Step 12: To map different modalities to a unified semantic space, linear projection or a multilayer perceptron is used to achieve dimension alignment.

[0023] ,

[0024] in, This represents the unified spatial features of the text modality after projection. The unified spatial features of the image modal after projection. It is a linear projection matrix of text features. It is a linear projection matrix of visual features. It is the text feature bias vector. It is a visual feature bias vector. This is a unified input dimension required for the backbone network of the large-scale policy model;

[0025] Step 13: Concatenate the aligned feature vectors into a paragraph-like input vector sequence according to the semantic structure:

[0026]

[0027] in, For the first A text feature vector after uniform spatial projection For the first A unified spatial projection of the image feature vector. These are special markers reserved in text instructions to indicate the insertion position of image modality information;

[0028] Step 2, Multimodal Fusion:

[0029] Step 21: The spliced ​​sequence Semantic modeling is performed by inputting a multi-layer Transformer backbone network: The Transformer uses a multi-head self-attention mechanism to interactively model multimodal inputs and generate context-sensitive hidden representations. ;

[0030] Step 22: The model's language head will hide the last layer of states. Mapping to the vocabulary space, the probability distribution of the next token is generated using Softmax:

[0031]

[0032] in, This represents the target token predicted by the model at the current time step. Indicates time step All previously generated historical token sequences, For the input visual image, The input text command sequence is as follows. It is the projection matrix of the language head output layer. For vocabulary size, Indicates time step The hidden state, This represents the output layer bias vector;

[0033] Step 23: For the generated sequence Its log-likelihood is defined as:

[0034]

[0035] This log probability is calculated synchronously during the model's forward propagation and used in the policy loss function:

[0036]

[0037] in, This indicates a reward signal. For the policy distribution of model parameterization, These are the learnable parameters of the model;

[0038] Step 3: Prediction of physical key points and gripper pose:

[0039] Step 31: Design key point distance rewards Predict key points of home objects:

[0040]

[0041] in, and These represent the predicted and actual keypoint coordinates, respectively. This is the scaling factor;

[0042] Step 32: Design gripper pose reward Predict the gripper pose:

[0043]

[0044] in, and These represent the predicted and actual gripper Y-axis vectors (gripping directions), respectively. and These represent the Z-axis vectors (orientation) of the grippers. and These represent the initial motion direction vectors, , , These are the weighting coefficients. Sensitivity to control directional error;

[0045] Step 33: Introduce orthogonal reward for grippers Ensure the orthogonality of the gripper coordinate system by forcing the Y-axis to be perpendicular to the Z-axis:

[0046]

[0047] in, The orthogonal penalty coefficient;

[0048] Step 34: Design motion direction rewards for different hinge types. Ensure that the initial direction of motion conforms to physical constraints:

[0049]

[0050] in, The translation direction vector, It is a radial unit vector;

[0051] Step 35: Force the model to output a standardized structure and design a formatted reward function. :

[0052]

[0053] Step 36: The overall reward function is set as follows:

[0054]

[0055] in, These are weighting coefficients;

[0056] Step 4: Predicting the trajectory of an object:

[0057] For predicting the motion trajectory of hinged objects, they are divided into translational and rotational objects, and a targeted reward function system is designed for each type of object to guide the model to generate physically reasonable, smooth, and accurate motion trajectories. The specific steps are as follows:

[0058] Step 41, Design Direction Alignment Reward This is used to ensure that the overall motion direction of the predicted trajectory is consistent with the object's inherent translation direction.

[0059]

[0060] in, It is the unit vector in the direction of translation. and These are the starting and ending points of the trajectory, respectively.

[0061] Step 42: Design linear rewards This is used to ensure that all trajectory points are distributed along a straight line.

[0062]

[0063] in, It is the number of inliers fitted. Given the total number of trajectory points, a straight line is fitted using the Random Sample Consensus (RANSAC) algorithm to evaluate the ratio of trajectory points to interior points of the fitted straight line;

[0064] Step 43: Design path similarity reward This is used to encourage predicted trajectories to resemble the shapes of actual trajectories.

[0065]

[0066] in, The distance scaling factor is used to predict the trajectory using Dynamic Time Warping (DTW) metric. With the actual trajectory The similarity between them;

[0067] Step 44: The total reward for predicting the trajectory of translational objects is:

[0068]

[0069] in, These are the weighting coefficients used to balance the importance of different constraints.

[0070] Step 45: Design the circular arc fitting reward This is used to excite the predicted trajectory points to be as close as possible to an ideal circular arc centered on the hinge axis.

[0071]

[0072] in, It is the fitted circle center. It is the fitting radius. It is the error scaling factor. To predict the coordinates of each point on the trajectory, This represents the number of trajectory points;

[0073] Step 46: Design Axial Alignment Bonus The direction of motion (movement direction vector) used to excite the end effector of the robotic arm is consistent with the tangent direction (i.e., the ideal instantaneous direction of motion) at each trajectory point:

[0074]

[0075] in, The reference point for the axis of rotation, The unit vector is the direction of the rotation axis;

[0076] Step 47: Design motion smoothness rewards This is used to encourage a smooth and continuous trajectory throughout.

[0077]

[0078] in, , for The tangential unit vector at time t, This is the scaling factor for smoothing.

[0079] Step 48: The total reward for predicting the trajectory of rotating objects is as follows:

[0080]

[0081] Among them, the weighting coefficient Used to adjust the contribution level of various rewards;

[0082] Step 49: To ensure that the model outputs standardized and parsable trajectory data, a format reward is designed. The trajectory prediction results are required to be output in a specific structure, and positive rewards are given for outputs that conform to this format specification, are complete, and can be correctly parsed. Conversely, punishment will be given:

[0083]

[0084] in, The number of trajectory points;

[0085] Step 5, Adaptive Strategy Selection:

[0086] Step 51: When a home operation task is received, the GRPO strategy selection mechanism is triggered through task semantic parsing and visual scene recognition. This mechanism is based on "task requirements - strategy matching" and dynamically selects suitable strategies according to the task type: if the core of the task is to accurately locate the operation position, the key point prediction strategy is automatically matched and the sub-strategy adapted to the visual features of the unknown object is called; if the core is to plan a safe and efficient movement path for the robotic arm to open cabinet doors and slide drawers, the trajectory prediction strategy is switched to and the sub-strategy that fits the constraints of the home scene is selected; if the task includes both positioning and path planning requirements, the two types of strategies will be activated simultaneously to form a linkage execution logic.

[0087] Step 52: After determining the target strategy, the VLM outputs operation-related features and instructions based on the selected strategy, driving the downlink unit to complete the initial operation attempt. During the process, the VLM will collect "strategy execution feedback data" in real time and input it into the GRPO reward calculation module. This module generates a targeted reward signal in combination with the strategy type. GRPO enters the relative advantage evaluation stage: first, the parallel effect of the currently executed target strategy and the alternative strategies is compared. Based on the feedback data collected by the VLM, the reward results of different strategies are obtained. Then, the relative advantage degree is calculated based on the target strategy reward. The initial direction and magnitude of the strategy gradient are generated according to the positive and negative signs and the magnitude of the advantage degree.

[0088] Step 53: To avoid excessive policy parameter updates causing VLM performance oscillations, a KL divergence constraint term is introduced to optimize the initial gradient: First, calculate the KL divergence value of the "current and updated policy parameter distributions" and determine whether the gradient magnitude exceeds the stability threshold. If it is less than the threshold, the initial gradient is retained; if it exceeds the threshold, the gradient is optimized by reducing the learning rate and adjusting the direction. In the case of two types of policy linkage scenarios, the gradient relationship also needs to be coordinated to avoid inconsistencies in VLM output features.

[0089] Step 54: After gradient optimization, GRPO passes the final gradient to the parameter update unit of VLM. The update unit first maps the gradient to the parameters of the feature extraction layer and semantic association layer of the corresponding strategy in VLM, and then fine-tunes the target parameters according to the optimized gradient direction and magnitude to ensure that the strategy-related features output by VLM are more in line with the needs of home operation.

[0090] Step 55: After the parameters are updated, VLM re-executes the hinge object operation task based on the updated parameters. GRPO restarts the "strategy selection - reward generation - gradient calculation" process. Through multiple iterations, the parameters are gradually optimized until VLM achieves the preset standards for key point prediction accuracy and trajectory prediction rationality in the unknown home object operation task. Finally, a closed-loop process of "strategy selection - reward feedback - parameter update - performance improvement" is formed.

[0091] Compared with the prior art, the present invention has the following advantages:

[0092] 1. This invention specifically addresses the problem of inaccurate hinge object recognition in home scenarios, demonstrating significant effectiveness in the core service scenarios of the embodied butler. Testing was conducted on three high-frequency interactive objects: washing machines, refrigerator doors, and drawers. The washing machine recognition accuracy improved from 78% with the traditional algorithm to 96%; the refrigerator door recognition accuracy improved to 76%; and the drawer recognition accuracy improved from 71% to 78%. This significantly enhances the embodied butler's understanding and manipulation accuracy of hinge objects, supporting its autonomous completion of home services such as opening and closing doors.

[0093] 2. In response to the collision problem that easily occurs when the intelligent butler robotic arm is operated, this invention adds a motion trajectory prediction module to continuously adjust the robotic arm's movements, thereby avoiding collisions with fixed structures. Attached Figure Description

[0094] Figure 1 This is a schematic diagram of the ManipLLM architecture;

[0095] Figure 2 A diagram showing the relationship between an embodied intelligent visual language action model for hinge joint manipulation.

[0096] Figure 3 Architecture diagram of an embodied intelligent visual language motion model for hinge joint manipulation;

[0097] Figure 4 A schematic diagram for key point prediction and gripper pose prediction of translational objects;

[0098] Figure 5 This diagram illustrates keypoint prediction and gripper pose prediction for rotating objects.

[0099] Figure 6 A schematic diagram illustrating the actual and predicted trajectories of translational objects;

[0100] Figure 7 A schematic diagram illustrating the actual and predicted trajectories of rotating objects;

[0101] Figure 8 Illustration of manipulating rotating objects Figure 1 ;

[0102] Figure 9 Illustration of manipulating rotating objects Figure 2 ;

[0103] Figure 10 A schematic diagram for predicting the trajectory of rotating objects;

[0104] Figure 11 Illustration of manipulating translational objects Figure 1 ;

[0105] Figure 12 Illustration of manipulating translational objects Figure 2 ;

[0106] Figure 13 A schematic diagram for predicting the trajectory of translational objects. Detailed Implementation

[0107] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0108] This invention provides an embodied intelligent visual language action model for hinge joint manipulation. The model includes an image-text multimodal feature matching module, a visual language model, an object keypoint prediction and gripper pose prediction module, an object motion trajectory prediction module, and an adaptive strategy selection module. Figure 2 and Figure 3 The functions of each module and the relationships between them are given below:

[0109] The image-text multimodal feature matching module is responsible for aligning input information from both text and image modalities, responding to home operation needs, and completing the fusion construction of multimodal commands by encoding information from different modalities into a unified vector representation space.

[0110] The visual language model, as the core processing unit of the system, is responsible for receiving multimodal instruction segments and performing high-level semantic understanding and decision generation. It provides a solution specifically for the problem of randomly selected operation points not matching the gripper in the shortcoming (1). The model finally outputs accurate predictions of key operation points of the robotic arm, including gripping position, axis of rotation, end effector pose and motion trajectory. Among them, the operation points abandon the random selection method and are preferentially fixed on the manipulable parts of hinged objects such as microwave oven handles and sliding door handles; if there is no obvious handle structure (such as some cabinet doors and windows), the axis of rotation is used as the reference, and points that are farther away from the axis of rotation and have higher mechanical efficiency are selected to ensure that the output operation points are fully adapted to the gripping shape and force angle of the gripper, which meets the operation requirements of the robotic arm on hinged objects in real scenarios.

[0111] The object key point prediction and gripper pose prediction module addresses the problems of poor model generalization and failure of cross-hinge type operation in the shortcomings (2). Based on the geometric and physical characteristics of household hinge objects such as microwave oven hinges and window hinges, the module clarifies the core operation parameters through modeling, and then combines the algorithm design to output key information. Finally, it derives the appropriate pose constraint reward function to provide decision support for the operation of the robotic arm. This module innovatively divides hinge objects into two major types: rotational (such as microwave ovens and windows) and translational (such as sliding doors and storage cabinet drawers). It completes the accurate type judgment by capturing object features through images, and then locks the corresponding operation key points: locking the center point of the handle for rotational objects and locking the key contact point of the handle for translational objects. Then, it calibrates the initially located points according to the structure and motion characteristics of the two types of objects to ensure that the model can still output effective operation strategies on hinge objects outside the training set, significantly improving the cross-type generalization ability.

[0112] The object motion trajectory prediction module focuses on solving the problems of the inability to track the operation process and the easy occurrence of jamming and collision between the robotic arm and the hinged object in the shortcomings (3). Combining the motion characteristics and operation requirements of household objects such as microwave oven rotation and drawer opening and closing, it integrates information from multiple dimensions to output accurate trajectories, providing support for the robotic arm to plan operation paths and avoid motion interference. The module first sorts out the object motion reference, trajectory shape and key parameter range, and obtains the current state and key point coordinates of objects such as microwave oven and drawer through visual perception as the trajectory starting reference; then it analyzes operation commands such as closing windows and opening doors to extract the endpoint parameters and clarify the termination conditions; finally, it associates the object motion constraints (such as the maximum opening and closing angle of microwave oven and the sliding stroke of drawer) to ensure that the trajectory does not exceed the physical range and forms a start-end-constraint information chain. On this basis, the trajectory path is calculated according to the starting and target states with the motion reference as a reference; the speed change is planned in combination with the operation requirements to generate a continuous motion trajectory, while avoiding interference with surrounding cabinets, walls and other structures, realizing the full tracking of the motion trajectory of the movable part of the hinged object, and completely solving the problem of jamming and collision between the robotic arm and the fixed parts of the hinge.

[0113] The adaptive strategy selection module manages and optimizes the entire task process. It adaptively selects appropriate strategies based on the different progress of the two core tasks: keypoint pose recognition and object trajectory prediction. It embeds the Group Relative Policy Optimization (GRPO) reinforcement learning optimization method into the training process of the visual language model. This mechanism replaces traditional absolute rewards with a group relative policy evaluation mechanism, improving the stability and convergence efficiency of strategy training. GRPO leverages the relative nature of the group to compare reward feedback from multiple sets of operation samples, such as grasping a microwave oven handle and pulling a drawer. In the keypoint pose recognition task, it prioritizes localization strategies with high feature matching to quickly locate the key points of rotating or translating hinged objects. In the object trajectory prediction task, it switches to a motion constraint adaptation strategy to ensure the continuity and collision avoidance of trajectory generation. This ensures the coordinated operation of the three core modules, ultimately forming a complete hinged object manipulation solution.

[0114] An embodied intelligent visual language motion model architecture for hinge joint control, such as Figure 3 As shown, the detailed process of each module is as follows:

[0115] (1) Image-text multimodal feature matching module

[0116] Text input involves the user entering home operation commands such as "turn on the microwave" or "close the window." The system then integrates and packages these commands with system prompts and model dialogue templates to form a complete text string command. Image input involves acquiring visual perception data and robotic arm pose perception data through sensor components: Visual perception data comes from images of home objects such as microwave ovens, washing machines, storage cabinet drawers, and windows captured by a monocular camera. These single images need to undergo processing such as size transformation, border pixel filling, and data format conversion to form an image format that the model can process; Robotic arm pose perception data is acquired by the robotic arm's own sensors, including the coordinates of the robotic arm next to the microwave oven and in front of the storage cabinet, the angles of each joint relative to the opening and closing of the window, and the opening and closing information of the gripper adapting to the storage cabinet drawer handle. After reading, this data is concatenated with the text command data such as "turn on the microwave" in string form.

[0117] Suppose the input data contains a text modality. With image modality Then the entire input set can be represented as: For information from different modalities, separate feature encoders are designed: ,in, and Let represent the feature extraction functions for the text modality and the image modality, respectively, and let represent their feature vectors. It is the dimension of the text feature vector. This refers to the dimension of the visual feature vector. To map different modalities to a unified semantic space, linear projection or a multilayer perceptron is used to achieve dimension alignment.

[0118] (1)

[0119] in, This represents the unified spatial features of the text modality after projection. The unified spatial features of the image modal after projection. It is a linear projection matrix of text features. It is a linear projection matrix of visual features. It is the text feature bias vector. It is a visual feature bias vector. This provides a unified input dimension required for the backbone network of the large-scale policy model. During the multimodal input construction stage, the aligned feature vectors are concatenated into a segmented input vector sequence based on semantic structure.

[0120] (2)

[0121] in, For the first A text feature vector after uniform spatial projection For the first A unified spatial projection of the image feature vector. These are special markers reserved in text instructions to indicate the insertion position of image modal information. For example, in the instruction "Open the washing machine door," the symbol... Corresponding image features of the washing machine's exterior or hinge area are inserted to achieve task-related semantic alignment.

[0122] Through the above steps, inputs from different modalities are mapped to a unified multimodal feature space, achieving dual alignment of dimensionality and semantic consistency. The resulting segmented multimodal input representation retains complementary features between modalities while also satisfying the input specifications of the main model, providing an efficient fusion representation foundation for multimodal task understanding and decision-making.

[0123] (2) Visual Language Model Module

[0124] In the visual language model stage, aligned visual and textual features are used as input. A Transformer structure is employed to deeply fuse multimodal information and model the context, enabling the model to understand image content, locate key objects, resolve spatial relationships, and perform task-related semantic reasoning based on language commands. Building upon this, the Visual Language Model (VLM) autoregressively generates probability distributions of action sequences, mapping language commands and visual observations into executable decision outputs, which directly serve as the policy network to provide policy outputs for subsequent reinforcement learning. Simultaneously, the VLM maintains the generalization ability of the pre-trained model, providing stable and iteratively updatable policy representations for GRPO optimization, making it a key module for achieving precise robot operation under language guidance.

[0125] In the multimodal fusion stage, the spliced ​​sequence Semantic modeling is performed by inputting a multi-layer Transformer backbone network: The Transformer uses a multi-head self-attention mechanism to interactively model multimodal inputs and generate context-sensitive hidden representations. The model's language head will hide the last layer of state. Mapping to the vocabulary space, the probability distribution of the next token is generated using Softmax:

[0126] (3)

[0127] in, This represents the target token predicted by the model at the current time step. Indicates time step All previously generated historical token sequences, For the input visual image, The input text command sequence is as follows. It is the projection matrix of the language head output layer. For vocabulary size, Indicates time step The hidden state, This represents the output layer bias vector. In GRPO training mode, the model needs to calculate the log probability of each generated token to support the loss calculation for the policy gradient. For the generated sequence... Its log-likelihood is defined as:

[0128] (4)

[0129] This log probability is calculated synchronously during the model's forward propagation and used in the policy loss function:

[0130] (5)

[0131] in, This indicates a reward signal. For the policy distribution of model parameterization, These are the learnable parameters of the model.

[0132] (3) Physical key points and gripper pose prediction module

[0133] In tasks involving hinged objects such as opening and closing washing machine doors, pushing and pulling storage cabinet drawers, and opening and closing windows, robotic arms need to accurately predict key interaction points and plan reasonable gripper poses to complete these complex home operations. However, existing methods often suffer from positional deviations when predicting key points, such as difficulty in accurately locating the center point of the washing machine door hinge or the contact point of the storage cabinet drawer handle. Furthermore, they lack physical constraints when outputting gripper poses, such as insufficient pose adaptation for window hinge rotation logic or mismatch between gripper angles and the sliding trajectory of the storage cabinet drawer. Ultimately, this leads to insufficient robotic arm operation precision (e.g., deviation when gripping the washing machine door handle) and inappropriate strategies (e.g., interference between the gripper and the window frame when opening a window).

[0134] To address the issues of keypoint prediction bias and pose inconsistency, this invention designs a reward function system to resolve these problems. This system guides the model to learn operational strategies that conform to the laws of mechanics through geometric constraints and physical priors, thereby improving the robotic arm's operational accuracy and strategy rationality regarding hinged objects. Figure 4 , Figure 5 As shown.

[0135] For keypoint prediction of home objects, a negative Euclidean distance penalty is converted into a reward to incentivize the model to predict points closer to reality. A differentiated reward mechanism is designed based on the object's motion type.

[0136] (6)

[0137] in, As a key point distance reward, and Represent the predicted and actual keypoint coordinates, respectively, and the scaling factor. Control the rate of reward decay and balance distance accuracy with physical plausibility.

[0138] To address the lack of physical plausibility and motion continuity in gripper pose prediction, we designed a multi-dimensional reward function system. This system guides the model to learn physically sound operating strategies through orientation alignment constraints, orthogonality guarantees, and motion plausibility verification, thereby improving the robotic arm's accuracy and stability in manipulating hinged objects.

[0139] Gripper pose prediction requires accurate estimation of three key direction vectors: gripper holding direction (Y-axis), gripper orientation (Z-axis), and initial motion direction. This invention designs a direction alignment reward based on cosine similarity:

[0140] (7)

[0141] in, For gripper position reward, and These represent the predicted and actual gripper Y-axis vectors (gripping directions), respectively. and These represent the Z-axis vectors (orientation) of the grippers. and These represent the initial motion direction vectors. Weighting coefficients. , , This highlights the difference in importance between different directions. Sensitivity to control directional error.

[0142] To ensure the orthogonality of the gripper coordinate system, this invention introduces an orthogonal constraint reward, forcing the Y-axis to be perpendicular to the Z-axis. This is the orthogonal penalty coefficient. The reward value is 1 when the Y-axis is completely perpendicular to the Z-axis, and drops to 0 when the two axes are parallel, effectively ensuring the normalization of the gripper coordinate system.

[0143] (8)

[0144] in, Orthogonal reward for grippers

[0145] For different hinge types, this invention designs differentiated motion rationality rewards to ensure that the initial motion direction conforms to physical constraints:

[0146] (9)

[0147] in, Rewards for the direction of exercise. The translation direction vector, This is a radial unit vector. Translation-type rewards encourage motion directions parallel to the translation axis, while rotation-type rewards encourage motion directions perpendicular to the radial direction.

[0148] Simultaneously, the model output is forced to have a standardized structure to ensure parsingability. For example, the format should include labels "<Thinking>" and "<Answer>", where the answer format should be "<Answer> Gripper contact point: (c1, c2), gripper movement direction: (y1, y2, y3), gripper X-axis direction: (x1, x2, x3), gripper Z-axis direction: (z1, z2, z3) <Answer>", and a reward function is designed accordingly.

[0149] (10)

[0150] The overall reward function is set as follows:

[0151] (11)

[0152] in, It is the weighting coefficient.

[0153] (4) Object motion trajectory prediction module

[0154] In trajectory prediction tasks for robotic arms manipulating objects (such as rotating microwave oven doors, sliding storage cabinet drawers, and hinged cabinet doors), there are problems such as large deviations between predicted and actual trajectories, and a lack of continuous and accurate tracking of the orientation of household objects. This is because existing trajectory prediction methods lack accurate adaptation to objects with different motion characteristics (rotation, translation, and hinged motion). For example, it is difficult to match the arc of a microwave oven door rotating around an axis or the linear pattern of a storage cabinet drawer sliding along a track. It is also difficult to continuously capture the orientation changes of hinged objects such as cabinet doors during operation, making it difficult for the predicted trajectory to conform to the actual motion patterns and intelligently adapt to the object's orientation.

[0155] To solve this problem, the model needs to accurately encode the geometric and physical characteristics of object motion (such as the rotation axis position of a microwave oven door, the length of a drawer slide rail) and the orientation changes of hinged objects such as cabinet doors. It also needs to guide the trajectory prediction to approximate the true trajectory and adapt to the orientation through a reward function. This invention improves upon these issues by enhancing trajectory and orientation features and aligning the trajectory with the true path and orientation, thus addressing the problems of inaccurate trajectory prediction and insufficient orientation tracking. For the prediction of hinged object motion trajectories, they are divided into translational objects (such as drawer slide rails) and rotational objects (such as cabinet doors and microwave oven doors), and a targeted reward function system is designed for each type of object to guide the model to generate physically reasonable, smooth, and accurate motion trajectories. This system ensures that the predicted trajectory conforms to the motion laws of the object through geometric constraints (such as drawer sliding stroke limits) and kinematic priors (such as the rotation angle range of cabinet doors). The motion of translational objects (such as drawers and sliding covers) is mainly a straight-line trajectory, and their reward function design focuses on directional consistency, path linearity, and similarity to the true trajectory, such as... Figure 6 As shown.

[0156] Orientation Alignment Reward This is used to ensure that the overall motion direction of the predicted trajectory is consistent with the object's inherent translation direction.

[0157] (12)

[0158] in, It is the unit vector in the direction of translation. and These are the starting and ending points of the trajectory, respectively.

[0159] Linear rewards All trajectory points are distributed along a straight line. The Random Sample Consensus (RANSAC) algorithm is used to fit the line, and the proportion of trajectory points to the interior points of the fitted line is evaluated. It is the number of inliers fitted. Total number of trajectory points:

[0160] (13)

[0161] Path similarity reward Encourage predicted trajectories to resemble the true trajectory shape, and use Dynamic Time Warping (DTW) as a metric for predicted trajectories. With the actual trajectory Similarity between them This is the distance scaling factor, used to control how sensitive the similarity is to distance:

[0162] (14)

[0163] Combining the above rewards, the total reward for predicting the trajectory of translational objects is:

[0164] (15)

[0165] in These are the weighting coefficients used to balance the importance of different constraints.

[0166] The motion of rotating objects (such as doors and covers) follows a circular trajectory and must maintain its geometric relationship with the axis of rotation. Its reward function emphasizes the circularity of the trajectory, axis alignment, and smoothness of the motion. Figure 7 As shown.

[0167] Circular fitting reward The incentive is to encourage the predicted trajectory point to be as close as possible to an ideal circular arc centered on the hinge axis. This reward is achieved by calculating the distance of the trajectory point from the theoretical arc plane and the radial distance deviation from the axis.

[0168] (16)

[0169] Among them, is the fitted center of the circle, is the fitted radius, is the error scaling factor, are the coordinates of each point on the predicted trajectory, is the number of trajectory points.

[0170] Axial alignment reward , which encourages the movement direction (the moving direction vector) of the end of the robotic arm to be consistent with the tangent direction (i.e., the ideal instantaneous movement direction) at each trajectory point, which is crucial for applying effective forces.

[0171] (17)

[0172] Among them, is the reference point of the rotation axis, is the unit vector in the direction of the rotation axis.

[0173] Motion smoothness reward , which encourages the entire trajectory to be smooth and continuous, avoiding sudden changes in direction or speed to ensure the stability of execution.

[0174] (18)

[0175] Among them, , is the tangential unit vector at time is the smooth scaling factor. Therefore, the total reward for trajectory prediction of rotating objects is as follows, and the weight coefficient is used to adjust the contribution degree of each reward.

[0176] (19)

[0177] To ensure that the model outputs standardized and parsable trajectory data, we design a format reward. It is required that the trajectory prediction result be output in a specific structure, for example: <Thought>Contact point continuous tracking, chain-of-thought reasoning process, predicting the path of key points of the object< / Thought>, <Answer>Point 1(a1, a2), Point 2(b1, b2), ……, Point N(n1, n2), 3 < N < 10 <Answer>. A positive reward is given to the output that conforms to this format specification, has complete data, and can be correctly parsed , and a penalty is given otherwise, is the number of trajectory points.

[0178] (20)

[0179] (5) Adaptive policy selection module

[0180] For different tasks, this invention proposes an adaptive strategy selection module. When receiving home operation tasks such as "opening the microwave oven door" or "pushing and pulling the storage cabinet drawer", the GRPO strategy selection mechanism is first triggered through task semantic parsing and visual scene recognition. This mechanism is based on "task requirements-strategy matching" and dynamically selects suitable strategies according to the task type: if the core of the task is to accurately locate the operation position of the microwave oven door handle, storage cabinet drawer handle, etc., the key point prediction strategy is automatically matched and the sub-strategy adapted to the visual features of the unknown object is called; if the core is to plan a safe and efficient movement path for the robotic arm to open the cabinet door and slide the drawer, the trajectory prediction strategy is switched to and the sub-strategy that fits the constraints of the home scene (such as avoiding surrounding cabinets and walls) is selected; if the task includes both positioning (such as finding window hinges) and path planning (such as the window opening and closing trajectory) requirements, the two types of strategies will be activated simultaneously to form a linkage execution logic.

[0181] After the strategy selection mechanism determines the target strategy, the VLM outputs operation-related features and instructions based on the selected strategy (such as the coordinates of the key points of the microwave oven door rotation and the sliding trajectory parameters of the storage cabinet drawer), driving the downlink unit to complete the initial operation attempt. During the process, the VLM collects "strategy execution feedback data" in real time and inputs it into the GRPO reward calculation module. This module generates a targeted reward signal based on the strategy type. GRPO then enters the relative advantage evaluation stage to replace the traditional absolute reward mechanism: first, the currently executed target strategy is compared with the alternative strategies in parallel (such as comparing the microwave oven door key point prediction strategy with the general positioning strategy). Based on the feedback data collected by the VLM, the reward results of different strategies are obtained. Then, the relative advantage degree is calculated based on the target strategy reward (positive value indicates that the target strategy is better, and negative value reflects inadequacy). The initial direction and magnitude of the strategy gradient are generated according to the sign and magnitude of the advantage degree.

[0182] To avoid performance oscillations in the VLM caused by excessively large policy parameter updates, a KL divergence constraint term is introduced to optimize the initial gradient: First, the KL divergence value of the "current and updated policy parameter distributions" is calculated, and it is determined whether the gradient magnitude exceeds a stability threshold. If it is less than the threshold, the initial gradient is retained; if it exceeds the threshold, the gradient is optimized by reducing the learning rate and adjusting the direction. For scenarios involving two linked policies, such as "locating window hinges + planning opening and closing trajectories," the gradient relationship needs to be further coordinated to avoid inconsistencies in the VLM output features. After gradient optimization, GRPO passes the final gradient to the VLM's parameter update unit. The update unit first maps the gradient to the parameters of the corresponding policy's feature extraction layer (e.g., recognizing the outline of home objects) and semantic association layer (e.g., associating window opening commands with hinge movement logic) in the VLM. Then, it fine-tunes the target parameters according to the optimized gradient direction and magnitude to ensure that the policy-related features output by the VLM better match the needs of home operations.

[0183] After the parameters are updated, VLM re-executes tasks such as "closing the microwave oven door" and "pushing and pulling the drawer" based on the updated parameters. GRPO restarts the "strategy selection - reward generation - gradient calculation" process, gradually optimizing the parameters through multiple iterations until VLM achieves the preset standards for key point prediction accuracy and trajectory prediction rationality in unknown home object operation tasks (such as new style storage cabinet drawers), ultimately forming a closed-loop process of "strategy selection - reward feedback - parameter update - performance improvement".

[0184] Example:

[0185] To make the objectives, technical solutions, and advantages of this invention clearer, the following will provide a detailed description of this embodiment in conjunction with natural scene images.

[0186] The system development platform is the Linux operating system, the GPU is an NVIDIA GeForce GTX TITAN X GPU, the program is written in Python 3.8 and uses the PyTorch 1.12 framework; the algorithm is deployed on the robotic arm.

[0187] The process of operating a microwave oven with a robotic arm is as follows: When the robotic arm of the smart home system is about to open the microwave oven, the entire process will proceed in the following steps, with each module working together to ensure the smooth completion of the operation.

[0188] First is the image-text multimodal matching module, which is fully adaptable to the home application scenarios of smart home systems. When a user issues the command "turn on the microwave," the system integrates the command with system prompts and dialogue templates to create a complete string command that better reflects the user's actual needs. Simultaneously, a monocular depth camera accurately captures images of target home objects such as the microwave, which are then resized and formatted to form visual perception data. When performing modal alignment between text and visual data, the module uses an independent feature encoder to convert them to a vector space of the same dimension. After adjusting according to the input dimension of the large model, the data is concatenated into a multimodal command segment based on pre-defined markers.

[0189] Next, the large language model processes the fused multimodal features. At this point, the smart home's strategy selector comes into play, addressing the diverse needs of tasks in a home setting. Homes may have rotating operations like opening a microwave oven, or sliding operations like opening a drawer. The strategy selector can dynamically choose the appropriate strategy from predicted key points of operation, gripper pose strategies, and predicted motion trajectory strategies, based on these specific home task requirements, ensuring that the smart home can respond efficiently to different home operation tasks.

[0190] Next is the key point prediction and gripper pose module. Based on the information output by the visual language model, the grasping operation points, gripper direction, robotic arm orientation, and movement direction of the smart home robot arm are clearly defined. Considering the differences in the size of microwave oven door handles and the position of drawer handles in the home, the smart home robot will accurately predict the key operation points of the gripper corresponding to the door handle based on the structural characteristics of different home objects. At the same time, it plans the direction in which the gripper fits against the door, the orientation of the robotic arm to avoid home obstacles such as microwave oven bodies or drawer cabinets, and the initial movement direction of the robotic arm.

[0191] Next is the object trajectory prediction module, which adapts to the safety operation needs of smart home systems in home scenarios. Based on the output of the visual language model and combined with the unique geometric structures of different types of objects—for example, microwave ovens have the physical characteristic of rotating doors around an axis, and storage cabinet drawers have the pattern of translating along slide rails—the predicted trajectory gradually approximates the actual sliding trajectory. Simultaneously, the process incorporates a path planning reward function, evaluating and optimizing the trajectory from multiple aspects such as the standardization of trajectory data format, the arc fitting degree of the microwave oven door rotation, the axial alignment of the drawer sliding, and the smoothness of the robotic arm's movement. This ensures both the accuracy of trajectory prediction and avoids collisions between the robotic arm or household objects and surrounding items, meeting the operational safety requirements of home scenarios.

[0192] Finally, after strategy selection and execution, the smart home assistant calculates the reward weights for different strategies based on the pose prediction reward function and the path planning reward function. Based on the relative advantages between the reward weights and the grouping strategies, the parameters of the visual language model are updated, and iterative optimization improves the model's accuracy and adaptability in handling home operation tasks. Simultaneously, KL divergence constrains the magnitude of strategy updates, preventing abrupt policy changes and ensuring the convergence and robustness of the model training process. This guarantees that the smart home assistant can reliably and stably perform various operations in home scenarios over the long term, continuously meeting users' home usage needs.

Claims

1. An embodied intelligent visual language action model for hinge joint control, characterized in that... The embodied intelligent visual language action model includes an image-text multimodal feature matching module, a visual language model, an object keypoint prediction and gripper pose prediction module, an object motion trajectory prediction module, and an adaptive strategy selection module, wherein: The image-text multimodal feature matching module is responsible for aligning input information from both text and image modalities, responding to home operation needs, and completing the fusion and construction of multimodal commands by encoding information from different modalities into a unified vector representation space. The visual language model is responsible for receiving multimodal instruction segments and performing high-level semantic understanding and decision generation, outputting accurate predictions of key points of robotic arm operation, including gripping position, axis direction, end effector pose and motion trajectory. The object key point prediction and gripper pose prediction module is responsible for using the geometric and physical characteristics of the home hinge object as a basis, clarifying the core operation parameters through modeling, and then combining the algorithm design to output key information, and finally deriving the appropriate pose constraint reward function to provide decision support for the operation of the robotic arm. The object motion trajectory prediction module is responsible for combining the motion characteristics and operational needs of the home objects, integrating information from multiple dimensions to output accurate trajectories, and providing support for the robotic arm to plan its operation path and avoid motion interference. The adaptive policy selection module is responsible for adaptively selecting appropriate policies based on the different progress of the two core tasks of key point pose recognition and object trajectory prediction. It embeds the GRPO reinforcement learning optimization method into the training process of the visual language model and improves the stability and convergence efficiency of policy training through a group relative policy evaluation mechanism.

2. A method for hinge joint manipulation using the embodied intelligent visual language action model as described in claim 1, characterized in that... The method includes the following steps: Step 1: Image-text multimodal feature matching: Step 11: Assume the input data contains a text modality. With image modality Then the entire input set is represented as: For information from different modalities, separate feature encoders are designed: ,in, and Let represent the feature extraction functions for the text modality and the image modality, respectively, and let represent their feature vectors. It is the dimension of the text feature vector. It is the dimension of the visual feature vector; Step 12: To map different modalities to a unified semantic space, linear projection or a multilayer perceptron is used to achieve dimension alignment. , in, This represents the unified spatial features of the text modality after projection. The unified spatial features of the image modal after projection. It is a linear projection matrix of text features. It is a linear projection matrix of visual features. It is the text feature bias vector. It is a visual feature bias vector. This is a unified input dimension required for the backbone network of the large-scale policy model; Step 13: Concatenate the aligned feature vectors into a paragraph-like input vector sequence according to the semantic structure: in, For the first A text feature vector after uniform spatial projection For the first A unified spatial projection of the image feature vector. These are special markers reserved in text instructions to indicate the insertion position of image modality information; Step 2, Multimodal Fusion: Step 21: Combine the spliced ​​sequence Semantic modeling is performed by inputting a multi-layer Transformer backbone network: The Transformer uses a multi-head self-attention mechanism to interactively model multimodal inputs and generate context-sensitive hidden representations. ; Step 22: The model's language head will hide the last layer of states. Mapping to the vocabulary space, the probability distribution of the next token is generated using Softmax: in, This represents the target token predicted by the model at the current time step. Indicates time step All previously generated historical token sequences, For the input visual image, The input text command sequence is as follows. It is the projection matrix of the language head output layer. For vocabulary size, Indicates time step The hidden state, This represents the output layer bias vector; Step 23: For the generated sequence Its log-likelihood is defined as: This log probability is calculated synchronously during the model's forward propagation and used in the policy loss function: in, This indicates a reward signal. For the policy distribution of model parameterization, These are the learnable parameters of the model; Step 3: Prediction of physical key points and gripper pose: Step 31: Design key point distance rewards Predict key points of home objects; Step 32: Design gripper pose reward Predict the gripper pose; Step 33: Introduce orthogonal reward for grippers To ensure the orthogonality of the gripper coordinate system, the Y-axis is forced to be perpendicular to the Z-axis; Step 34: Design motion direction rewards for different hinge types. This ensures that the initial direction of motion conforms to physical constraints. Step 35: Force the model to output a standardized structure and design a formatted reward function. ; Step 36: The overall reward function is set as follows: in, These are weighting coefficients; Step 4: Predicting the trajectory of an object: For predicting the motion trajectory of hinged objects, they are divided into translational objects and rotational objects, and a targeted reward function system is designed for each type of object to guide the model to generate physically reasonable, smooth and accurate motion trajectories. Step 5, Adaptive Strategy Selection: Step 51: When a home operation task is received, the GRPO strategy selection mechanism is triggered through task semantic parsing and visual scene recognition. This mechanism is based on "task requirements - strategy matching" and dynamically selects suitable strategies according to the task type: if the core of the task is to accurately locate the operation position, the key point prediction strategy is automatically matched and the sub-strategy adapted to the visual features of the unknown object is called; if the core is to plan a safe and efficient movement path for the robotic arm to open cabinet doors and slide drawers, the trajectory prediction strategy is switched to and the sub-strategy that fits the constraints of the home scene is selected; if the task includes both positioning and path planning requirements, the two types of strategies will be activated simultaneously to form a linkage execution logic. Step 52: After determining the target strategy, the VLM outputs operation-related features and instructions based on the selected strategy, driving the downlink unit to complete the initial operation attempt. During the process, the VLM will collect "strategy execution feedback data" in real time and input it into the GRPO reward calculation module. This module generates a targeted reward signal in combination with the strategy type. GRPO enters the relative advantage evaluation stage: first, the parallel effect of the currently executed target strategy and the alternative strategies is compared. Based on the feedback data collected by the VLM, the reward results of different strategies are obtained. Then, the relative advantage degree is calculated based on the target strategy reward. The initial direction and magnitude of the strategy gradient are generated according to the positive and negative signs and numerical values ​​of the advantage degree. Step 53: To avoid excessive policy parameter updates causing VLM performance oscillations, a KL divergence constraint term is introduced to optimize the initial gradient: First, calculate the KL divergence value of the "current and updated policy parameter distributions" and determine whether the gradient magnitude exceeds the stability threshold. If it is less than the threshold, the initial gradient is retained; if it exceeds the threshold, the gradient is optimized by reducing the learning rate and adjusting the direction. In the case of two types of policy linkage scenarios, the gradient relationship also needs to be coordinated to avoid inconsistencies in VLM output features. Step 54: After gradient optimization, GRPO passes the final gradient to the parameter update unit of VLM. The update unit first maps the gradient to the parameters of the feature extraction layer and semantic association layer of the corresponding strategy in VLM, and then fine-tunes the target parameters according to the optimized gradient direction and magnitude to ensure that the strategy-related features output by VLM are more in line with the needs of home operation. Step 55: After the parameters are updated, VLM re-executes the hinge object operation task based on the updated parameters. GRPO restarts the "strategy selection - reward generation - gradient calculation" process. Through multiple iterations, the parameters are gradually optimized until VLM achieves the preset standards for key point prediction accuracy and trajectory prediction rationality in the unknown home object operation task. Finally, a closed-loop process of "strategy selection - reward feedback - parameter update - performance improvement" is formed.

3. The hinge joint control method according to claim 2, characterized in that... The key point distance reward The expression is: in, and These represent the predicted and actual keypoint coordinates, respectively. This is the scaling factor.

4. The hinge joint control method according to claim 2, characterized in that... The gripper pose reward The expression is: in, and These represent the predicted and actual gripper Y-axis vectors, respectively. and These represent the Z-axis vectors of the grippers, and These represent the initial motion direction vectors, , , These are the weighting coefficients. Sensitivity to control directional error.

5. The hinge joint control method according to claim 2, characterized in that... The gripper orthogonal reward The expression is: in, This is the orthogonal penalty coefficient.

6. The hinge joint control method according to claim 2, characterized in that... The direction of movement reward The expression is: in, The translation direction vector, It is a radial unit vector.

7. The hinge joint control method according to claim 2, characterized in that... The format reward function The expression is: 。 8. The hinge joint control method according to claim 2, characterized in that... The specific steps of step 4 are as follows: Step 41: Design Direction Alignment Reward This is used to ensure that the overall motion direction of the predicted trajectory is consistent with the object's inherent translation direction. Step 42: Design linear rewards This is used to excite all trajectory points to be distributed along a straight line; Step 43: Design path similarity reward This is used to encourage predicted trajectories to resemble the shapes of actual trajectories. Step 44: The total reward for predicting the trajectory of translational objects is: in, These are the weighting coefficients used to balance the importance of different constraints; Step 45: Design the circular arc fitting reward This is used to excite the predicted trajectory points to be as close as possible to an ideal circular arc centered on the hinge axis. Step 46: Design Axial Alignment Bonus This is used to ensure that the motion direction of the robotic arm's end effector is consistent with the tangent direction at each trajectory point; Step 47: Design motion smoothness rewards This is used to encourage a smooth and continuous trajectory. Step 48: The total reward for predicting the trajectory of rotating objects is as follows: Among them, the weighting coefficient Used to adjust the contribution level of various rewards; Step 49: To ensure that the model outputs standardized and parsable trajectory data, a format reward is designed. The trajectory prediction results are required to be output in a specific structure, and positive rewards are given for outputs that conform to this format specification, are complete, and can be correctly parsed. Conversely, punishment will be given: in, This represents the number of trajectory points.

9. The hinge joint control method according to claim 8, characterized in that... The direction alignment reward The expression is: in, It is the unit vector in the direction of translation. and These are the starting and ending points of the trajectory, respectively; The linear reward The expression is: in, It is the number of inliers fitted. Given the total number of trajectory points, a straight line is fitted using a random sampling consensus algorithm to evaluate the ratio of trajectory points to interior points of the fitted straight line; The path similarity reward The expression is: in, The distance scaling factor is used to predict the trajectory using the Dynamic Time Warping (DTW) metric. With the actual trajectory The similarity between them.

10. The hinge joint control method according to claim 8, characterized in that... The axial alignment reward The expression is: in, The reference point for the axis of rotation, It is the unit vector in the direction of the rotation axis; The motion smoothness reward The expression is: in, , for The tangential unit vector at time t, This is the smooth scaling factor.

Images