Control method based on information-enhanced visual action language model and reinforcement learning

Abstract

The application discloses a control method based on information-enhanced visual motion language model and reinforcement learning, which comprises the following steps: analyzing a job instruction, triggering multi-source sensing collection; performing time delay calibration and credibility weighted fusion on original data to form a pre-processing information pool. Operation descriptors are generated in combination with instructions and pre-processing information. A cross-modal contact consistency field is constructed by using pre-processing data, and a fine operation executable index is calculated. Nominal actions are generated, and after online correction and safety constraint monitoring, safety actions are executed. Feedback data are collected after execution, and the contact field, the executable index and the local contact stiffness matrix are updated online, and an abnormality index is generated. The task completion state is continuously judged, and if it is not completed, the abnormality index is adjusted for correction, and the feedback-updating-correction is repeated until the task completion result is output. By using the application, the alignment accuracy and contact safety of fine operation are improved. The application can be widely applied to the technical field of robot control.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a control method based on information-enhanced visual action language model and reinforcement learning. Background Technology

[0002] At the intersection of robotics and artificial intelligence, visual language models are propelling embodied intelligent robot manipulation to the forefront. This is also one of the core topics in current embodied intelligence research, aiming to enable robots to understand natural language commands, perceive their surroundings visually, and autonomously plan and execute a series of actions. The core of this approach is a powerful Visual Motion Language Model (VLM) pre-trained on massive amounts of image-text pairing data, serving as the robot's "intelligent hub," enabling it to perform semantic parsing and task reasoning in open scenes. In practice, existing sophisticated manipulation schemes typically input visual images, natural language commands, and the robot's own state into the visual motion language model, which then directly outputs end-effector pose increments, joint velocities, or gripper control parameters to drive the robotic arm to perform actions.

[0003] However, existing methods typically only stitch together or fuse visual, tactile, and force perception at the feature layer, failing to address cross-modal misalignment caused by differences in sampling frequencies and communication delays between cameras, force sensors, tactile arrays, and controllers. When the robot has already made contact but the visual frame still corresponds to the pre-contact state, the model is prone to outputting incorrectly oriented or excessively large corrective actions. Summary of the Invention

[0004] In view of this, in order to solve the technical problem that existing robot control methods often fail to calibrate multimodal data, resulting in low control accuracy, this invention proposes a control method based on information-enhanced visual action language model and reinforcement learning. This method includes the following steps: Task Parsing and Data Aggregation: First, the work instructions are parsed, and the acquisition of multi-source sensor information is triggered simultaneously. Data Preprocessing: Time synchronization compensation (latency calibration) is performed on the various types of raw data collected, and differentiated weighted fusion is performed based on the reliability of each data source to form a unified preprocessed information pool. Operation Intent Generation: Combining the task instructions and preprocessed information, operation descriptors are generated to guide action planning. Contact State Modeling and Feasibility Assessment: A cross-modal contact consistency field is constructed using preprocessed data, and a fine-grained operational executability index is quantitatively calculated based on this field. Nominal Action Planning: Integrating the operation descriptors, preprocessed data, contact field model, and executability index, a preliminary action sequence (nominal action) is generated. Safety Optimization and Execution: The nominal actions are corrected online, and under the monitoring of safety constraints, a safe and reliable action instruction is finally output and executed. Execution Feedback Acquisition: After the safe action is executed, sensor feedback data on the system status and external environment are acquired in real time. Model Dynamic Update and Anomaly Detection: Based on feedback data, the cross-modal contact consistency field, fine-grained operational feasibility index, and local contact stiffness matrix are adjusted online, while anomaly indicators are generated to characterize deviations. Task Progress Evaluation and Iterative Correction: The system continuously determines whether the current task has reached completion. If not, the action strategy is adjusted to correct deviations based on the anomaly indicators. This feedback-update-correction process is repeated until a preset termination criterion is met, and finally, a confirmation result of task completion is output.

[0005] Based on the above scheme, this invention provides a control method based on an information-enhanced visual action language model and reinforcement learning. Through event-driven cross-modal time-delay calibration, visual, tactile, force-sensing, and proprioceptive states are aligned to the same physical moment, reducing erroneous contact judgments caused by differences in sampling frequency and communication delay. Furthermore, through a cross-modal contact consistency field with spatial attributes and a fine-operation executable index, target geometry, contact direction, force control state, tactile stability, slip risk, and perception credibility are converted into continuously computable indices, enabling the system to distinguish between normal contact, skew, slip, jamming, and sensing anomalies, thereby improving the alignment accuracy, contact safety, anti-disturbance capability, and anomaly recovery efficiency of fine operations. Attached Figure Description

[0006] Figure 1 This is a flowchart of the steps of a control method based on an information-enhanced visual action language model and reinforcement learning according to the present invention. Detailed Implementation

[0007] Besides the technical issues mentioned in the background, existing methods mostly use semantic similarity, failure flags, target distance, or a single contact force threshold to determine the state. They lack continuous, computable indicators that can simultaneously describe geometric alignment, contact normal consistency, pressure distribution stability, slip risk, and perceived uncertainty. Therefore, they are difficult to accurately distinguish physical states such as no contact, normal compliant contact, jamming, skewed insertion, and slippage. Reinforcement learning strategies often directly generate complete actions or score candidate trajectories as a whole, lacking online identification of local contact stiffness changes and mechanisms for compensating for geometric errors, force errors, and material compliance changes. They are prone to oscillations or error accumulation when faced with switching between soft and hard materials, changes in hole-axis clearance, or sudden changes in assembly friction. Anomaly detection is usually based on fixed thresholds or consecutive failures, and cannot utilize the model residuals between the predicted and actual contact responses for anomaly attribution. This easily leads to frequent VLM replanning calls or repeated attempts by the underlying strategy even when structured instructions have failed.

[0008] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0009] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0010] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0011] Unless the context explicitly indicates an exception, words such as "a," "an," "a kind," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list; a method or apparatus may also include other steps or elements. An element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0012] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0013] Reference Figure 1 This is a flowchart illustrating an optional example of the control method based on information-enhanced visual action language model and reinforcement learning proposed in this invention. This method can be applied to computer devices, and the control method proposed in this embodiment may include, but is not limited to, the following steps: Step S1: Obtain task instructions and collect multi-source data.

[0014] Step S2: Perform time delay calibration and quality weighting preprocessing on the multi-source data.

[0015] Step S3: Generate an operation descriptor based on the task instructions and preprocessed data.

[0016] Step S4: Construct a cross-modal contact consistency field based on the preprocessed data and calculate the fine-grained operational executability index.

[0017] Step S5: Generate nominal action.

[0018] Step S6: Modify the nominal action and execute the safety action in conjunction with safety constraints.

[0019] Step S7: Collect feedback data after performing the safety action.

[0020] Step S8: Update the cross-modal contact consistency field, executable exponent, and local contact stiffness matrix based on the feedback data, and construct anomaly indicators.

[0021] Step S9: Determine the task completion status and make corrections based on abnormal indicators until the termination conditions are met, and output the task completion result.

[0022] In some feasible embodiments, step S1 specifically includes: The robot receives fine-grained operation instructions in the form of natural language or voice, and simultaneously acquires RGB images, depth maps, robotic arm joint positions and speeds, six-dimensional force / torque at the end effector, pressure distribution of the tactile array, gripper status, and historical actions.

[0023] The raw observations for the t-th control period are represented as follows: .in, For color images, For depth map, This refers to the robot's main body state. For the end-effector six-dimensional force / torque, For tactile pressure matrix, This is the action from the previous control cycle. Each sensor retains its original sampling timestamp and data quality status.

[0024] In some feasible embodiments, step S2 specifically includes: Constructing motion event sequences from visual and proprioceptive states respectively For example, the optical flow amplitude of the target region, the rate of change of end pose, or a weighted sum of the two; Constructing sequences of physical contact events from force or touch data. Examples include the rate of change of normal force, the rate of change of pressure centroid, or the rate of change of contact area.

[0025] For the For each physical modality, within the allowable time delay range, search for the time offset that maximizes its correlation with visual motion events: In the above formula, positive Δτ represents reading the original data of the physical mode at t+Δτ when calculating the alignment characteristics at the unified time t.

[0026] After determining the time delay, the original data is interpolated and resampled.

[0027] Specifically, using the unified physical time t corresponding to the robot control cycle as the target time reference, and based on the determined time offsets of each mode... From the m-th physical mode in Adjacent time points are selected from nearby raw sampling data, and aligned data corresponding to a unified physical time t are obtained through linear interpolation, spline interpolation, or other time resampling methods. This transforms visual, tactile, force, and proprioceptive state data with different sampling frequencies and communication delays to a unified control time axis.

[0028] The residual time error refers to the residual time deviation that still exists between the effective sampling time of the m-th mode and the unified physical time t after time offset compensation and interpolation resampling are completed. This residual time deviation can be determined based on the time distance between the adjacent original sampling points used for resampling and the target time, the correlation peak deviation after event sequence calibration, or a combination of both.

[0029] Based on sensor saturation Frame dropping degree and remaining time error Calculate the quality coefficient And form quality-weighted features: in, This represents the primitive feature of the m-th mode. The 'Norm' represents the features after calibration and quality weighting, 'Interp' represents the normalization process, and 'Interp' represents interpolation resampling. During the training phase, contact event center sampling is further employed to maintain a relative balance among phases such as approach, initial contact, stable contact, slippage, jamming, and task completion in the training batch. Simultaneously, low-quality modes are randomly masked, and quality coefficients are used as model inputs, ensuring the model can still operate even when individual sensors malfunction.

[0030] In some feasible embodiments, step S3 specifically includes: VLM generates a structured contact operation descriptor based on the task instructions, the visual region of the target object, the depth point cloud, and the current robot state. The descriptor includes at least the operation object and target object, target pose, expected contact area, target surface normal, allowed contact direction, target normal force range, tangential force limit, allowed slip risk, propulsion speed limit, phase termination condition, and abnormal recovery candidate actions.

[0031] For example, for the task of "inserting a plug into a socket", the descriptor sets the target angle between the plug axis and the socket axis, the insertion depth, the target normal force range, the allowable lateral force, and the allowable continuous jamming time.

[0032] In addition, operation descriptors Instead of being directly sent to the robot controller, the control quantities are used as conditional parameters for subsequent contact state modeling, nominal motion generation, and safety constraints. Specifically, the target pose, expected contact area, and target surface normal are used to determine the geometric alignment and contact normal consistency of candidate spatial points in step S4; the target normal force range, tangential force upper limit, and allowable slip risk are used to calculate the force control matching degree, slip safety degree, and cross-modal contact consistency field; the target pose, allowable contact direction, and propulsion speed upper limit, together with the preprocessed multimodal features, cross-modal contact consistency field, and fine operation executability index, serve as conditional inputs for information enhancement VLA in step S5 to generate nominal motions; the target normal force range, tangential force upper limit, allowable contact direction, and propulsion speed upper limit are further used to construct the safe feasible region in step S6; the stage termination condition and anomaly recovery candidate actions are used for task completion judgment and anomaly recovery in step S9.

[0033] In some feasible embodiments, step S4 specifically includes: Candidate space points are selected in the target operating space. And calculate the normalized geometric distance from the depth point to the target contact surface using cloud computing. and visual surface normal vector ; The actual contact normal is estimated based on the six-dimensional force / torque and tactile pressure distribution. Normal force Tangential force Pressure center of mass and perceived uncertainty ; The spatial location of the target contact surface is determined based on the depth point cloud, and the current actual contact point is estimated by combining the current pose of the end effector, the effective contact area in the haptic array, and the six-dimensional force / torque information, denoted as . To give force and tactile information spatial properties for estimating the point of contact. A local spatial kernel function is constructed around the center, such that the actual contact normal vector, local contact force, pressure centroid change, and perceived uncertainty are calculated according to the candidate spatial point p and the estimated contact point. The spatial distance between them is mapped to the target operating space.

[0034] Define the slip risk at candidate spatial point p as a combination of the rate of change of the pressure centroid and the utilization rate of the friction cone: Where μ is the friction coefficient of the contact surface, σ is the Sigmoid function, and ε is a positive number to avoid a denominator of zero.

[0035] Based on this, a cross-modal contact consistency field is constructed: in, Representing a spatial point The normalized geometric distance to the target contact surface. Represents the visual surface normal vector. This represents the actual contact normal vector. This represents the normalized error between the target force range and the actual local contact force. Representing a spatial point The slip risk at the point is a combination of the rate of change of the pressure center of mass and the utilization rate of the friction cone. It represents perceived uncertainty. The closer it is to 1, the more consistent the target geometry, contact direction, contact force, tactile stability, and perceived reliability are. A lower value indicates that the candidate action may be skewed, stuck, slipped, or have an observational distortion.

[0036] Based on the above cross-modal observation information, the geometric alignment degree, force control matching degree, contact stability degree, slip safety degree, and observation reliability are calculated respectively.

[0037] The degree of geometric alignment is determined based on the normalized geometric distance from the candidate spatial point to the target contact surface, the deviation between the current end-effector pose and the target pose, and the angle between the visual surface normal and the target surface normal. It is used to characterize the spatial alignment state between the robot end-effector and the target contact area.

[0038] The force control matching degree is determined based on the normalized error between the actual normal force obtained by the six-dimensional force / torque sensor and the target normal force range in the operation descriptor; the force control matching degree is higher when the actual normal force is within the target force range, and the lower the index is when it deviates from the target force range.

[0039] The degree of contact stability is determined based on at least one of the following: the rate of change of contact area in the tactile pressure array, the rate of change of pressure centroid, and the degree of change of pressure distribution. When the contact area and pressure centroid remain stable, the index is high; when the pressure distribution changes rapidly or the pressure centroid continues to drift, the index decreases.

[0040] The degree of slippage safety is based on the aforementioned slippage risk. It is determined that the slip risk is negatively correlated with the slip risk; when the rate of change of the pressure center of mass increases or the tangential force relative to the friction cone boundary increases, the slip risk increases and the corresponding slip safety level decreases.

[0041] The observation reliability is determined based on the quality coefficients of each sensing mode and the sensing uncertainty. The observation reliability decreases when sensor saturation, frame loss, or increased remaining time error occur. Thus, the above five sub-indicators respectively reflect the current executable status of the five aspects: geometry, force, touch, slip, and sensing quality.

[0042] Further normalize the geometric alignment degree, force control matching degree, contact stability degree, slip safety degree, and observation reliability to , , , and The fine-grained operational executability index is obtained by using weighted geometric fusion. : Strictly defined as between 0 and 1. When any critical capability approaches an unexecutable state... Rapid descent. VLA will Local encoding of the contact consistency field as an action condition: Normal advancement is permitted at higher elevations; During descent, reduce propulsion speed, increase compliance, and expand safety margin. Mass coefficients are used uniformly. This indicates that, in order to avoid conflict with executable sub-indicators Confusing.

[0043] In some feasible embodiments, step S5 specifically involves the information-enhanced VLA receiving a structured contact operation descriptor, quality-weighted multimodal features, contact consistency field local coding, and... Output nominal continuous action a_t^0. The nominal motion can be the end-effector six-dimensional pose increment, joint velocity, gripper control quantity, or a combination thereof.

[0044] In some feasible embodiments, step S6 specifically includes: During the contact phase, the end-effector pose increment With contact force / torque increment Establish local linear relationships K can be set as a 3×3 translational stiffness matrix or a 6×6 pose-force / moment stiffness matrix depending on the control space. A sliding window with a forgetting factor, a time-continuous regularization term, and symmetric positive semidefinite constraints is used to estimate the local contact stiffness matrix. The semi-positive definite constraint in the above formula avoids obtaining a stiffness estimate that contradicts the physical characteristics of passive contact. Based on the target pose error... Target contact force error and contact Jacobi matrix Generate model compensation actions.

[0045] For irreversible or ill-conditioned stiffness matrices, a damped pseudo-inverse is used: Residual reinforcement learning strategy It only outputs bounded correction values ​​and does not directly replace the nominal VLA action.

[0046] Define the normalized prediction residuals of the stiffness model High model residuals and low executable exponents increase the proportion of residual strategies, while high perceived uncertainty reduces the control of residual strategies and decreases the overall action magnitude through a safety scaling factor. The model compensation and residual reinforcement learning actions are fused together as follows: This dual-gating method avoids mistakenly amplifying reinforcement learning correction actions when the perceived quality is poor.

[0047] After superimposing the VLA nominal motion and the corrected motion, instead of directly issuing the motion to the robot, the safest motion closest to the original motion is solved under constraints of joint limits, velocity, collision distance, maximum contact force, and friction cone. in, This represents the joint changes that occur during action a within one control cycle. This represents the minimum distance between the predicted trajectory and the obstacle. and These represent the predicted normal force and tangential force, respectively. The optimization can be solved using quadratic programming, a control barrier function, or a differentiable optimization layer. The robot controller executes the algorithm. It also collects multi-source feedback for the next control cycle.

[0048] In some feasible embodiments, step S8 specifically includes: The system updates the contact consistency field based on the newly acquired RGB-D, tactile, force, and body states. Executable Index And the local contact stiffness matrix. The contact force increment is predicted using an online stiffness model, and the normalized residual between the actual and predicted increments is calculated. Simultaneously calculate the normalized visual target region drift. and Degradation amount, constructing cumulative anomaly indicators: All input terms in the above formula are normalized to the range of 0 to 1. Among them, and These are the local correction threshold and the semantic reconstruction threshold, respectively, and 0 < < When the cumulative abnormal indicators < If the current deviation is considered to be within the normal control fluctuation range, the current action strategy will continue to be executed.

[0049] when ≤ < When the semantic recognition results of the target object, target contact area, and task stage remain stable, it is determined to be an action-level anomaly. An action-level anomaly indicates that the current structured contact operation descriptor is still valid, and the anomaly mainly originates from local pose deviation, contact force deviation, slippage, local stiffness changes, or other physical state changes that can be corrected by the underlying control strategy. For action-level anomalies, the underlying controller performs local rollback, attitude fine-tuning, compliance parameter adjustment, or force control parameter adjustment.

[0050] when ≥ A semantic-level anomaly is identified when significant changes are detected in the target object, target region, target surface normal, or task phase, or when the structured contact operation descriptor fails to meet the current observation state within a preset number of consecutive control cycles. This semantic-level anomaly indicates that the current structured contact operation descriptor may have become invalid, requiring re-interpretation of the higher-level task or regeneration of the contact strategy. In this case, the anomaly type, low-consistency region, direction of local contact stiffness change, and history of failed actions are uploaded to the VLM to regenerate or correct the structured contact operation descriptor.

[0051] In some feasible embodiments, step S9 specifically includes: After a semantic-level anomaly is triggered, VLM corrects the target surface normal, contact area, force control range, propulsion speed, or recovery steps, and returns to S3 to regenerate the structured contact operation descriptor. For action-level exceptions, retain the original descriptor and return to S4 to update the consistency field and subsequent actions.

[0052] Based on the termination condition, target pose error, insertion depth, and contact force stability in the descriptor. The system determines whether the task is complete based on the visual target status. If the task is not completed, it returns S3 or S4 according to the exception level; if the termination condition is met, it outputs the task completion result.

[0053] Model training is completed before online execution and can be divided into three stages. The first stage uses demonstration data to train information to enhance the nominal action prediction capability of VLA; the second stage uses synchronously acquired visual, tactile, force, and ontological data to train the contact consistency field prediction head and the next-state dynamics prediction head; the third stage fixes or reduces the VLA backbone learning rate and trains a bounded residual reinforcement learning strategy on simulation environment and safe real machine data. VLM can use cue constraints or small-scale supervised fine-tuning to generate structured contact operation descriptors with fixed fields.

[0054] Residual reinforcement learning states include quality-weighted multimodal features, The system employs local encoding of the contact consistency field, online stiffness estimation, nominal motion, and historically corrected motion; motion consists only of bounded residuals. The reward function incentivizes improved task progress and executability, while penalizing overforce, slippage, motion amplitude, jitter, and collision. in, A normalized progress increment defined based on insertion depth, target distance, rotation angle, or other task load; Normalized penalty for exceeding the target force range.

[0055] The joint objective of information-enhanced VLA, consistent field prediction head, dynamic prediction head, and residual strategy is: in, Loss due to cloning of nominal actions and behaviors. To predict loss in contact with the uniform field, To predict the loss for the pose and force response in the next cycle, The residual strategy is reinforced by the learning loss. During training, the nominal action and auxiliary prediction tasks are pre-trained separately, and then jointly optimized with a small learning rate to avoid the reinforcement learning gradient from destroying the existing general semantics and action capabilities of VLA.

[0056] Based on all embodiments of the above methods, the beneficial effects of the present invention also include: adapting to changes in material hardness, assembly gaps, and friction states through online contact stiffness identification and model compensation; reducing instability caused by directly using reinforcement learning to generate complete actions by correcting only the nonlinear parts that are difficult for the model to describe through bounded residual reinforcement learning; improving the safety of fragile object handling and precision assembly by explicitly limiting joints, collision distances, normal forces, and tangential friction constraints through safety constraint action projection; and achieving hierarchical triggering of normal continuation, action-level local correction, and semantic-level descriptor reconstruction by constructing cumulative anomaly indices through normalized stiffness prediction residuals, visual drift, and executable exponent degradation, thereby reducing unnecessary large model calls and invalid retries.

[0057] Based on the above method flow, the present invention also provides a specific example of the specific execution process of inserting a plug into a socket: After receiving the instruction to "insert the plug into the socket," the robot locates the plug and socket positions using RGB-D images. The VLM (Virtual Machine Library) generates a structured contact operation descriptor containing the plug axis, socket axis, target insertion depth, maximum axial force, maximum lateral force, and recovery motion. The system performs time-delay calibration on the images, tactile array, and six-dimensional force sensor, calculating the geometric alignment as the end effector approaches the socket.

[0058] After the initial contact, the system calculates the contact consistency field based on the surface normal of the hole, the actual force direction, the tactile pressure centroid, and the tangential force. When the plug is slightly deflected, the normal consistency decreases, the lateral force increases, and the pressure centroid moves to one side, causing Q_t to decrease accordingly. The system reduces the advance speed and determines whether the current state is normal edge contact or rigid jamming through online stiffness identification.

[0059] If the predicted force increment is basically consistent with the actual force increment, the model compensation generates small-angle oscillation and retraction actions based on the pose error and force error; if friction or hole gap causes significant nonlinearity, the residual reinforcement learning correction ratio is increased. All actions are executed after being projected with joint, collision distance, maximum force, and friction cone constraints.

[0060] When the plug enters the hole and the insertion depth reaches the target value, the lateral force decreases, the contact force returns to stability, and... When the value continuously exceeds the completion threshold, the system determines that the insertion is complete. If the model residual and visual drift increase simultaneously after multiple corrections, the VLM re-identifies the hole orientation and updates the structured contact operation descriptor, instead of allowing the underlying strategy to retry indefinitely.

[0061] A control system based on an information-enhanced visual action language model and reinforcement learning, comprising: The data acquisition module is used to execute step S1; The data preprocessing module is used to perform step S2; The descriptor generation module is used to execute step S3; The index calculation module is used to perform step S4; The action generation module is used to execute step S5; The action execution module is used to execute step S6; The feedback acquisition module is used to execute step S7; The abnormal indicator construction module is used to execute step S8; The task completion judgment module is used to execute step S9.

[0062] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0063] A control device based on an information-enhanced visual action language model and reinforcement learning: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the control method based on an information-enhanced visual action language model and reinforcement learning as described above.

[0064] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0065] A storage medium storing processor-executable instructions, which, when executed by a processor, are used to implement the control method based on an information-enhanced visual action language model and reinforcement learning as described above.

[0066] The content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0067] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A control method based on an information-enhanced visual-action language model and reinforcement learning, characterized in that, Includes the following steps: Obtain task instructions and collect data from multiple sources; The multi-source data is subjected to time delay calibration and quality weighting preprocessing to obtain preprocessed data; Based on the task instructions and the preprocessed data, an operation descriptor is generated; Based on the preprocessed data, a cross-modal contact consistency field is constructed and a fine-grained operation executability index is calculated. A nominal action is generated based on the operation descriptor, the preprocessed data, the cross-modal contact consistency field, and the fine-grained operation executable index. The nominal action is modified, and a safety action is performed in conjunction with safety constraints.

2. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 1, characterized in that, Also includes: Collect feedback data after performing safety actions; The cross-modal contact consistency field, executable exponent, and local contact stiffness matrix are updated based on the feedback data, and anomaly indicators are constructed. Determine the task completion status and make corrections based on the aforementioned abnormal indicators until the termination conditions are met, then output the task completion result.

3. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 2, characterized in that, The step of determining the task completion status and making corrections based on the abnormal indicators until the termination condition is met, and then outputting the task completion result, specifically includes: When it is determined that the task is not completed and the task is a semantic-level anomaly, correct the target surface normal, contact area, force control range, and propulsion speed, and return to the operation descriptor generation step; When it is determined that the task is not completed and the task is an action-level exception, retain the original operation descriptor and return to the step of building a cross-modal contact consistency field; Continue until the termination condition is met, then output the task completion result.

4. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 1, characterized in that, The step of performing time delay calibration and quality weighting preprocessing on the multi-source data to obtain preprocessed data specifically includes: Constructing motion event sequences from visual data and ontological states; Constructing a sequence of physical contact events from tactile data; Calculate the time offset based on the motion event sequence and the physical contact time sequence; The quality coefficient is calculated based on sensor saturation, frame drop rate, and remaining time error. By combining the time offset and the quality coefficient, calibrated and quality-weighted features are generated to obtain preprocessed data.

5. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 1, characterized in that, The step of constructing a cross-modal contact consistency field and calculating a fine-grained operational executability index based on the preprocessed data specifically includes: Candidate spatial points are selected in the target operating space, and the normalized geometric distance from the spatial points to the target contact surface and the visual surface normal vector are calculated based on the depth point cloud. Estimate the actual contact normal vector, normal force, tangential force, pressure centroid, and perception uncertainty based on the six-dimensional torque and tactile pressure distribution; Based on the normalized geometric distance from the spatial point to the target contact surface, the visual surface normal vector, the actual contact normal vector, the contact force, the pressure centroid, and the perceived uncertainty, a cross-modal contact consistency field is constructed. The degree of geometric alignment, the degree of force control matching, the degree of contact stability, the degree of slip safety, and the reliability of observation are normalized, and a fine-operation executability index is obtained by weighted geometric fusion.

6. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 5, characterized in that, The formula for the cross-modal contact uniformity field is as follows: in, Representing a spatial point The normalized geometric distance to the target contact surface. Represents the visual surface normal vector. This represents the actual contact normal vector. This represents the normalized error between the target force range and the actual local contact force. Representing a spatial point The slip risk at the point is a combination of the rate of change of the pressure center of mass and the utilization rate of the friction cone. It represents perceived uncertainty.

7. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 6, characterized in that, The step of modifying the nominal action and performing a safety action in conjunction with safety constraints specifically includes: Estimate the local contact stiffness matrix and combine it with the target pose error, target contact force error and basic Jacobian matrix to generate a compensation model; The residual correction is calculated by combining the model prediction residual and the fine-grained operational feasibility index. The model compensation and the residual correction are fused together, and the nominal action is corrected to obtain the corrected action; The modified action is projected onto the safe and feasible domain and executed.

8. The control method based on information-enhanced visual action language model and reinforcement learning according to claim 7, characterized in that, The step of fusing the model compensation and the residual correction is expressed by the following formula: in, This refers to a fusion correction action that modifies the nominal action. This represents the safety scaling factor determined based on the multimodal perceived uncertainty. The gating coefficients of the residual reinforcement learning strategy are shown. This represents the model compensation action obtained based on the local contact stiffness model. This represents a residual reinforcement learning policy with parameter φ in state. The residual action of the output, clip(·) represents the clipping function. This represents the maximum allowable correction magnitude for residual reinforcement learning actions.

9. A control system based on an information-enhanced visual-action language model and reinforcement learning, characterized in that, include: The data acquisition module is used to obtain task instructions and collect data from multiple sources; The data preprocessing module is used to perform time delay calibration and quality weighting preprocessing on the multi-source data to obtain preprocessed data; The descriptor generation module is used to generate an operation descriptor based on the task instruction and the preprocessed data; The index calculation module constructs a cross-modal contact consistency field based on the preprocessed data and calculates the fine-grained operationally executable index. The action generation module is used to generate nominal actions based on the operation descriptor, the preprocessed data, the cross-modal contact consistency field, and the fine-grained operation executable index. The action execution module is used to modify the nominal action and execute a safety action in conjunction with safety constraints.

Images