Compliant control method and system based on semantic anchoring and frequency domain pulse residual error

Abstract

The application discloses a compliant control method and system based on semantic anchoring and frequency domain pulse residual error, first, a semantic-geometric anchoring field is constructed by using a visual language model and CLIP feature alignment, the uncertainty of visual positioning is converted into the confidence radius of a road point, and an energy optimal B-spline reference trajectory containing jerk minimization and visual relaxation constraint is generated accordingly; secondly, a customized mother wavelet is constructed based on the second-order under-damped mechanical characteristics of a robot body, a pulse adaptive continuous wavelet transform is performed on a real-time current signal, and weak contact features are extracted in a matched filtering manner; finally, time domain motion trends and frequency domain contact features are input into a double-flow gated residual network, a fusion weight is dynamically adjusted through an adaptive gating mechanism, and a torque residual error is output for compliant compensation. The application has clear physical interpretability, and significantly improves the semantic understanding ability and contact stability of the robot in complex operations.

Description

Technical Field

[0001] This invention relates to the field of robot control and artificial intelligence, and in particular to a robot compliant control method based on semantic-geometric anchored visual language model (VLM) and pulse adaptive frequency domain residual. Background Technology

[0002] With the rapid development of embodied intelligence technology, robots are moving from structured industrial environments to unstructured home and commercial service scenarios. In these complex scenarios, robots not only need semantic perception capabilities to understand natural language instructions and recognize unknown objects, but also need motion control capabilities to ensure compliance and stability during physical contact. Therefore, researching a robot operation method that can efficiently integrate high-level semantic perception and low-level motion control is of great significance.

[0003] Early robot manipulation primarily relied on traditional visual servoing and model-based impedance control methods. These methods typically assume a known or static environment and depend on manually designed feature points or precise object geometry models. However, traditional methods struggle to handle open-vocabulary natural language commands and often fail when faced with objects of unknown shape or uncertain position due to a lack of semantic understanding. Furthermore, traditional impedance control, mainly based on time-domain force / potential feedback, responds slowly to high-frequency collision and impact signals, making it difficult to achieve the same level of sensitive tactile reflexes as humans.

[0004] In recent years, with breakthroughs in visual language models and large-scale language models, many robot control algorithms based on large models have been proposed. For example, some models attempt to directly map images and text into robot motion commands through end-to-end networks. While these methods significantly improve the robot's semantic understanding capabilities, they suffer from the following significant drawbacks: First, the uncertainty of visual perception is not effectively utilized. Existing VLM models, such as Grounding DINO, often have jitter or positioning errors in their output bounding boxes. Directly using their center points as absolute motion target points leads to unsmooth robot trajectories, increasing mechanical wear and control risks. Existing trajectory planning algorithms (such as fifth-order polynomial interpolation or traditional B-splines) typically only consider point constraints, lacking a mechanism to dynamically adjust trajectory compliance based on visual confidence. Second, there is a lack of frequency domain analysis for contact dynamics. The physical interaction between the robot and its environment, especially rigid collisions, inherently contains rich frequency domain information. Most existing deep learning-based policy networks only process time-domain position and velocity signals, or directly use general short-time Fourier transforms for simple frequency domain feature extraction. However, the mechanical collision signals of robots are usually manifested as the impulse response of a second-order underdamped system. Common frequency domain analysis tools are not compatible with the waveform of this physical signal, which can lead to serious energy leakage. This makes it difficult for the network to extract weak contact features from the noise, resulting in untimely torque compensation or overshoot.

[0005] However, no system currently exists that can simultaneously address the two core challenges of uncertain planning in visual perception and high-frequency contact detection based on physical priors. Compared to simply relying on large, black-box, data-driven models, a pressing technical challenge is how to combine matched filter theory from signal processing with modern control optimization algorithms to construct a robot operating system with clear physical interpretability and adaptability to unstructured environments. Summary of the Invention

[0006] To address the issues of trajectory jitter caused by positioning uncertainty in visual perception and unstable contact control due to the inability of traditional frequency domain analysis to effectively extract rigid collision features in current robot operation methods, this invention proposes a compliant control method based on semantic anchoring and frequency domain pulse residuals. This method has clear physical interpretability and can significantly improve the robot's semantic understanding and contact stability in complex tasks.

[0007] According to one aspect of the present invention, a compliant control method based on semantic anchoring and frequency domain pulse residuals is provided, comprising:

[0008] Acquire RGB images and text commands of the target operation scene, and construct a semantic-geometric anchoring field by aligning visual language models with CLIP features;

[0009] Based on the constructed semantic-geometric anchoring field, the uncertainty of visual positioning is transformed into the confidence radius of waypoints, thereby generating an energy-optimal baseline trajectory that includes visual uncertainty constraints.

[0010] Based on the real-time acquired robot joint state data, time-domain motion feature flow and frequency-domain contact feature flow are obtained. A customized wavelet transform algorithm based on the robot's mechanical damping characteristics is used to perform pulse adaptive frequency-domain feature extraction on the frequency-domain contact feature flow to obtain the contact feature time-frequency map.

[0011] The time-domain motion feature stream and the contact feature time-frequency map are input into a dual-stream gated residual network to deduce and calculate the torque residual compensation value. The torque residual compensation value is then fused with the feedforward torque of the reference trajectory to drive the robot joint motors to execute.

[0012] As a further technical solution, RGB images and text instructions of the target operation scene are acquired, and a semantic-geometric anchoring field is constructed by aligning visual language models with CLIP features, including:

[0013] Open vocabulary detection is performed on RGB images using a visual language model, and the bounding boxes of target objects are output.

[0014] The CLIP model is used to calculate the feature similarity between text instructions and pixels within the bounding box, and an operation heatmap is generated.

[0015] Based on the peak regions and dispersion of the heatmap, a set of spatial waypoints is output. and its corresponding confidence radius ,in With heat map at waypoints The variances at each location are positively correlated.

[0016] As a further technical solution, generating an energy-optimal baseline trajectory that includes visual uncertainty constraints includes:

[0017] Spatial waypoints are fitted using B-spline curves, and control points are solved by minimizing the energy loss function.

[0018] ,

[0019] in, For the planned joint trajectory. For accelerometer, it represents the impact energy of the system; For spatial waypoints, The confidence radius corresponding to the spatial waypoint; The balance coefficient is n; n is the number of spatial waypoints. To prevent tiny quantities with a denominator of zero; The control points of the B-spline curve to be optimized; the smoothness of the trajectory constrained by the first term of the energy loss function, and the geometric consistency between the trajectory constrained by the second term and the visual observations. The smaller the value, the better the trajectory. The stronger the anchoring effect.

[0020] As a further technical solution, the construction of the customized wavelet transform algorithm includes:

[0021] The rigid collision process of the robot is modeled as the impulse response of a second-order underdamped system, and its time-domain expression is:

[0022] ,

[0023] in, The inherent damping ratio of the robot, The natural angular frequency, The frequency of the damped oscillation. It is the amplitude coefficient;

[0024] based on Constructing a custom wavelet transform algorithm for:

[0025] ,

[0026] in, This is the term for removing the mean. is a Gaussian window function to ensure tight support in the time domain, and C is the energy normalization coefficient.

[0027] As a further technical solution, based on the aforementioned customized wavelet transform algorithm, the current signal is processed... Perform continuous wavelet transform, the expression is:

[0028] ,

[0029] in, As a scale factor, The translation factor is... The output is a time-frequency diagram of the contact characteristics; this transformation process is equivalent to constructing a matched filter for the collision signal in the frequency domain.

[0030] As a further technical solution, the dual-stream gated residual network adopts a heterogeneous dual-stream structure, and its implementation process includes:

[0031] Current joint position of the robot ,speed and reference trajectory error Input to the time-domain feature encoding module, output time-domain feature vector ;

[0032] Contact feature time-frequency diagram Input frequency domain feature encoding module, output frequency domain contact feature vector ;

[0033] The gating weight is calculated based on the energy intensity of the frequency domain contact feature vector. The dual-flow features are weighted and fused, and the torque residual compensation value is output based on the fusion result.

[0034] As a further technical solution, the control law calculation formula that integrates the torque residual compensation value and the feedforward torque of the reference trajectory is as follows:

[0035] ,

[0036] in, The command torque sent to the joint motor; This is the torque residual compensation value; The feedforward torque, calculated based on the dynamic model, is obtained from the generated baseline trajectory parameters. and PD control gain; As the reference joint angular velocity, This is the reference joint angular acceleration.

[0037] As a further technical solution, a staged reinforcement learning strategy is adopted to optimize the parameters of the dual-stream gated residual network, including:

[0038] In the first stage, the adaptive gating unit is frozen, the gating weight is set to 0, and the network is trained using only time-domain streaming data to enable it to have basic trajectory tracking capabilities.

[0039] In the second stage, the adaptive gating unit is unfrozen, random rigid body collision interference is introduced into the simulation environment, and the network parameters are optimized using the near-end policy optimization algorithm. The reward function is designed as a weighted sum of the position tracking error term and the contact force control term, so that the network learns to adjust compliance using the torque residual compensation value at the moment of contact.

[0040] According to one aspect of the present invention, a compliant control system based on semantic anchoring and frequency domain pulse residuals is provided, comprising:

[0041] The perception planning unit is used to perform semantic-geometric anchoring and energy-optimal baseline trajectory generation, and outputs baseline trajectory parameters.

[0042] The signal processing unit has a built-in custom wavelet transform algorithm for pulse adaptive feature extraction of current sensor signals and outputs a contact feature time-frequency diagram.

[0043] The dual-stream inference unit loads a pre-trained dual-stream gated residual network to calculate torque residual compensation values ​​based on state data.

[0044] The motion control execution unit is used to control the movement of the robot's joint motors by combining the feedforward torque and the torque residual compensation value.

[0045] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the compliant control method based on semantic anchoring and frequency domain pulse residuals.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] This invention employs a closed-loop control logic of "perception-planning-execution," combining matched filter theory in signal processing with modern control optimization algorithms to construct a robot compliant control method and system with clear physical interpretability and adaptability to unstructured environments. It can simultaneously solve two core problems: uncertain planning in visual perception and high-frequency contact detection based on physical priors. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic flowchart of a compliant control method based on semantic anchoring and frequency domain pulse residuals provided in an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the system architecture and data flow provided in an embodiment of the present invention.

[0051] Figure 3 This is a schematic diagram of the structure of a dual-stream gated residual network provided in an embodiment of the present invention.

[0052] Figure 4 This is a schematic diagram of the customized wavelet construction and frequency domain feature extraction process provided in an embodiment of the present invention. Detailed Implementation

[0053] This invention provides a compliant control method based on semantic anchoring and frequency domain pulse residuals. Its purpose is to address the problems in current robot manipulation methods, such as trajectory jitter caused by positioning uncertainty in visual perception, and unstable contact control due to the inability of traditional frequency domain analysis to effectively extract rigid collision features. This invention employs a closed-loop control logic of "perception-planning-execution," and its technical solution is as follows:

[0054] Step 1: Construction and Uncertainty Modeling of Semantic-Aware Data Stream. To quantitatively describe the uncertainty of visual perception, this invention constructs a semantic-geometric anchoring field. The system first uses the Visual Language Model (VLM) to perform open-vocabulary detection on RGB images, outputting the bounding boxes of target objects; then, it uses the CLIP model to calculate text feature vectors. Image feature map The cosine similarity is calculated by comparing the feature similarity between the text instruction and the pixels within the bounding box, thereby generating an operation heatmap. Here, H and W represent the height and width of the heatmap, respectively. Finally, Gaussian fitting is used to extract spatial waypoints for the peak regions of the heatmap. And calculate the confidence radius based on the local variance of the heatmap. :

[0055] ,

[0056] in, The local variance of the heatmap near the peak represents the dispersion of the location. Peak response intensity; This is the normalization coefficient. With heat map at waypoints The variances at each location are positively correlated. The larger the value, the more ambiguous the visual language model's localization of the point, and the more relaxed the physical constraints should be. This step ultimately outputs a set of spatial waypoints. and its corresponding confidence radius .

[0057] Step 2: Energy-optimal relaxation programming based on B-splines. This invention constructs a k-order B-spline curve as the baseline trajectory. Its mathematical expression is:

[0058] ,

[0059] in, For the control points to be optimized, The basis functions are defined using the Cox-deBoor recursive formula, where j = 0, 1, 2, ..., m. To balance motion smoothness and visual anchoring constraints, this invention constructs and solves the following energy functional optimization problem, i.e., solving for the control points by minimizing the energy loss function:

[0060] ,

[0061] in, To prevent tiny quantities with a denominator of zero, For the planned joint trajectory. For accelerometer, it represents the impact energy of the system; For the spatial waypoints output in step 1, This corresponds to the confidence radius; The first term of the formula constrains the smoothness of the trajectory, and the second term constrains the geometric consistency between the trajectory and the visual observation. The smaller the value, the better the trajectory. The stronger the anchoring effect, the better. This formula introduces... As an adaptive penalty weight: when visual confidence is high ( When the trajectory is forcibly pulled towards the waypoint. When the confidence level is low, the optimizer automatically sacrifices geometric accuracy in exchange for minimizing the jerk term, thereby eliminating the impact of visual jitter on the mechanical body from a mathematical perspective.

[0062] Step 3: Physically driven custom wavelet transform. This invention does not use a general wavelet basis for the pulse adaptive frequency domain feature extraction, but instead constructs a custom wavelet that matches the robot's mechanical collision response. Specifically, for the non-stationary signal at the moment of robot contact, this invention designs a custom wavelet transform algorithm that satisfies physical isomorphism.

[0063] First, the rigid collision process of the robot is modeled as the impulse response of a second-order underdamped system. That is, a second-order underdamped impulse response model is established based on the mechanical impedance characteristics of the robot joints. :

[0064] ,

[0065] in, The inherent damping ratio of the robot, The natural angular frequency, For the damped oscillation frequency, the parameter and It was identified by applying a step torque to the robot's end effector.

[0066] based on Constructing a custom wavelet transform algorithm, i.e., constructing a custom mother wavelet. It must satisfy the wavelet admissibility condition, that is Furthermore, it possesses finite energy in the frequency domain. Therefore, this invention constructs the following customized mother wavelet:

[0067] ,

[0068] in, This is the term for removing the mean. A Gaussian window function is used to ensure tight support in the time domain. This mother wavelet is then used to analyze the current signal. Perform continuous wavelet transform:

[0069] ,

[0070] Where a is the scaling factor and b is the translation factor. This is the time-frequency diagram of the contact characteristics output. Mother wavelet The complex conjugate of , since the customized mother wavelet constructed in this invention is a real function, here Numerically equivalent to The physical essence of this transformation is to perform matched filtering in the time-frequency domain, which can significantly improve the signal-to-noise ratio of weak collision signals that are submerged in noise.

[0071] Step 4: Heterogeneous dual-stream gated residual network architecture and design principles.

[0072] The dual-stream gated residual network adopts a heterogeneous dual-stream structure, including: a temporal feature encoding module, which adopts a multilayer perceptron structure, and whose input is the current joint position of the robot. ,speed and reference trajectory error Output time-domain feature vector The frequency domain feature encoding module adopts a one-dimensional convolutional neural network structure, and the input is the contact feature time-frequency map obtained in step 3. Output frequency domain contact feature vector The adaptive gating unit calculates the gating weights based on the energy intensity of the frequency domain characteristics. And weighted fusion of the two stream features, where When detected When the energy in a specific frequency band exceeds a threshold The trend is toward 1, and the dominance of network reasoning shifts to frequency domain tactile perception.

[0073] This invention designs a specific network architecture to handle heterogeneous data streams, and its detailed parameters are as follows:

[0074] ① Temporal Feature Encoding Stream: Used to process low-frequency, large-amplitude rigid body motion states. It employs a 3-layer fully connected network, with the input layer dimension... Including joint position ,speed and reference trajectory error The number of hidden layer nodes are respectively The activation function used is ReLU, which improves inference speed through sparse activation. This results in the output of a temporal feature vector. .

[0075] ② Frequency Domain Feature Encoding Stream: Used to process high-frequency, transient contact force oscillation features. A three-layer one-dimensional convolutional neural network is employed, utilizing its translation invariance to capture local patterns of the contact signal along the time axis, unaffected by the absolute moment of contact. The input is a time-frequency slice of a specific frequency band output from step 3. Specifically, the first convolutional layer has a kernel size k=5, stride s=1, and number of channels C=32; the second convolutional layer has a kernel size k=3, stride s=1, and number of channels C=64; and the third convolutional layer has a kernel size k=3, stride s=1, and number of channels C=128. Max pooling is applied between each layer. Its output is a frequency domain contact feature vector. .

[0076] ③ Adaptive Gating Unit: Used for dynamically arbitrating the weights of the two-stream features. It specifically employs an attention mechanism with Sigmoid activation, as shown in the following formula:

[0077] ,

[0078] in, Learn the frequency domain signal representing a rigid collision; This is the gated bias term. When a collision occurs, the CNN extracts... The activation value increases significantly, making The network output is mainly determined by the frequency domain flow, thus achieving a millisecond-level compliant response to collisions.

[0079] The dual-stream gated residual network described in this invention is a network with optimized parameters, and its parameter optimization process employs a staged reinforcement learning strategy:

[0080] The first stage, cold start training: freeze the adaptive gating unit, set α=0, and train the network using only time-domain streaming data to enable it to have basic trajectory tracking capabilities;

[0081] The second stage is frequency domain injection training: the adaptive gating unit is unfrozen, random rigid body collision interference is introduced into the simulation environment, and the network parameters are optimized using the PPO (Proximal Policy Optimization) algorithm. The reward function is designed as a weighted sum of the position tracking error term and the contact force control term, so that the network learns to adjust compliance using Δτ at the moment of contact.

[0082] Step 5, Terminal execution and fusion control.

[0083] The final control law combines model-driven feedforward and data-driven residuals, as shown in the following formula:

[0084] ,

[0085] in, The command torque sent to the joint motor; This is the torque residual compensation value; The feedforward torque, calculated based on the dynamic model, is obtained from the generated baseline trajectory parameters. and PD control gain; As the reference joint angular velocity, This is the reference joint angular acceleration.

[0086] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0087] This invention uses the example of a compliant gripping operation on thin-walled glassware to illustrate the technical solution of this invention.

[0088] This embodiment constructs a robotic experimental system consisting of a 7-DOF collaborative robotic arm, an RGB-D camera, and a two-finger end effector. The task scenario is set as follows: on a cluttered experimental table, identify and grasp the handle of a thin-walled glass beaker placed in a random position. This task requires the robot to have extremely high visual semantic understanding capabilities (distinguishing between the beaker wall and the handle) and millisecond-level compliant response capabilities at the moment of contact to prevent the beaker from breaking.

[0089] Please see Figure 1 and Figure 2 This embodiment provides a compliant control method based on semantic anchoring and frequency domain pulse residuals. The specific execution process is as follows:

[0090] Step 1: Construction of semantically aware data stream and uncertainty modeling. The robot camera acquires RGB images of the current scene (resolution...). The system receives the text command "Grab the beaker handle". To address the uncertainty of visual positioning, the system performs semantic-geometric anchoring:

[0091] ① Coarse localization: Call the visual language model to perform open vocabulary detection and output the bounding box covering the entire beaker;

[0092] ② Fine-grained analysis: The bounding box region is cropped and input into the CLIP model. The feature similarity between the text instruction handle and the image pixels is calculated to generate a resolution of [resolution value missing]. Operational heatmap ;

[0093] ③ Anchoring parameter calculation: Extract the peak points of the heat map as spatial waypoints. Calculate the heatmap at the peak point. Normalized variance within the neighborhood Set normalization coefficients The confidence radius of the waypoint is calculated according to the formula. The radius This intuitively reflects the fuzzy range that the visual language model considers "probably here, but not entirely certain".

[0094] Step 2: Energy-optimal relaxation programming system based on B-spline. A baseline trajectory is generated using a 5th-order B-spline curve. The optimizer constructs the energy functional. The balance coefficient is set. tiny amount During the solution process, due to the confidence radius at the handle... The size is relatively large (1.5cm). The optimization algorithm automatically relaxes the geometric constraints at this point, generating a trajectory that transitions slightly smoothly near the handle, rather than a trajectory that forcibly creates an angle. This effectively reduces the motor jerk and avoids high-frequency vibration of the robotic arm when it approaches the glass beaker.

[0095] Step 3: The dual-domain shunt robotic arm in its physical state begins to execute motion. The underlying controller acquires joint current data at a frequency of 1kHz. and joint position data The data stream is split before entering the controller: Enter the time domain branch to maintain basic trajectory tracking; Enter the frequency domain branch to monitor potential collisions.

[0096] Step 4, Physics-driven Custom Wavelet Transform and Frequency Domain Feature Extraction (see...) Figure 4To accurately detect potential minor collisions between the robotic arm's end effector and the beaker wall, this embodiment pre-constructs a custom wavelet:

[0097] ① Parameter identification: During the experimental preparation stage, the natural frequency of the robotic arm's end effector was measured through a tapping experiment. (Approximately 19Hz), damping ratio ;

[0098] ② Wavelet construction: Substitute the above parameters into the formula to construct a customized mother wavelet with specific attenuation characteristics. ;

[0099] ③ Real-time conversion: for the acquired current signal Perform continuous wavelet transform. The system focuses on monitoring the wavelet coefficients corresponding to the scale factor 'a' in the 15Hz-50Hz frequency band, generating a time-frequency map of contact characteristics. .

[0100] In practice, when the robotic arm's fingertip lightly touches the glass wall, ordinary current sensors only display random noise, but... It instantly exhibits a high-energy bright spot around 19Hz, achieving high signal-to-noise ratio extraction of weak contacts.

[0101] Step 5, Reasoning and Decision Making in Heterogeneous Two-Stream Gated Residual Networks, please see... Figure 3 The data is simultaneously input into the heterogeneous two-stream network:

[0102] Temporal flow: The input dimension is 12 (6 joint positions + 6 joint velocities), the network structure is FC(128) -> ReLU -> FC(256) -> ReLU -> FC(64), and the output is temporal features. At this point, because the collision was minor, the time-domain error had not yet accumulated significantly. They tend to maintain their original rigid trajectory.

[0103] Frequency domain stream: Input is a 50ms window. Slicing. The network structure is: Conv1D(k=5, c=32) -> MaxPool -> Conv1D(k=3, c=64) -> Conv1D(k=3, c=128). CNN keenly captures... The bright spot features in the output high-response frequency domain features. .

[0104] Adaptive gating: The gating unit detects... Energy burst, weights of the Sigmoid output It quickly rose from 0.1 to 0.95.

[0105] Strategy Output: Feature Fusion The network outputs compliance compensation torque after being fed into the strategy head. .

[0106] Step 6: The terminal execution and fusion control final control law takes effect. Reference trajectory. The attempt to drive the robotic arm forward (potentially crushing the beaker) is still underway, but the residual torque generated in step 5... This generates a reverse virtual damping force, which partially counteracts the feedforward torque. Physically, this manifests as the robotic arm exhibiting a human-like reflex, automatically releasing force and fine-tuning its posture the instant it contacts the glass wall, ultimately sliding along the beaker wall and successfully grasping the handle without damaging the beaker.

[0107] To achieve the aforementioned network performance, a phased reinforcement learning strategy is adopted:

[0108] Environment setup: Using the Isaac Gym simulation environment, 4096 robot environments were created in parallel.

[0109] Phase 1 (Cold Start): Setting the Learning Rate The training process is performed for 500 iterations. The reward function contains only the position tracking error term. This stage Forced lock to 0.

[0110] Phase Two (Frequency Domain Injection): Thawing Rigid body obstacles are randomly generated in the environment, and a contact reward is introduced. ,in The network was trained for 2000 iterations. During this stage, the network learned to construct bright spots in the frequency domain. The mapping relationship of output unloading torque.

[0111] The implementation of the various embodiments of the present invention is based on programmed processing through a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a compliant control system based on semantic anchoring and frequency domain pulse residuals. This system is used to execute the compliant control method based on semantic anchoring and frequency domain pulse residuals in the above method embodiments.

[0112] The system includes: a perception and planning unit for performing semantic-geometric anchoring and energy-optimal baseline trajectory generation, and outputting baseline trajectory parameters; a signal processing unit with a built-in custom wavelet transform algorithm for pulse adaptive feature extraction of current sensor signals, and outputting a contact feature time-frequency map; a dual-stream inference unit that loads a trained dual-stream gated residual network for calculating torque residual compensation values ​​based on state data; and a motion control execution unit for controlling the movement of robot joint motors by combining feedforward torque and torque residual compensation values.

[0113] The compliant control system based on semantic anchoring and frequency domain pulse residual provided in this invention addresses the failure scenario of GNSS positioning systems in tunnels during autonomous driving. It employs several modules and, through a closed-loop control logic of "perception-planning-execution," combined with matched filter theory in signal processing and modern control optimization algorithms, constructs a robot compliant control system with clear physical interpretability and adaptability to unstructured environments. This system can simultaneously solve the two core problems of uncertain planning in visual perception and high-frequency contact detection based on physical priors.

[0114] It should be noted that the system embodiments provided by the present invention are used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The only difference is that corresponding functional modules are set. The principle is basically the same as that of the above system embodiments provided by the present invention. As long as those skilled in the art can improve the modules in the above system embodiments by referring to the specific technical solutions in other method embodiments and combining technical features to obtain corresponding technical means and technical solutions composed of these technical means, on the basis of the above system embodiments, and on the premise of ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments.

[0115] Based on the same inventive concept as any of the foregoing embodiments, this embodiment of the invention also provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the compliant control method based on semantic anchoring and frequency domain pulse residuals.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] In summary, this invention discloses a compliant control method based on semantic anchoring and frequency domain pulse residuals. This method aims to address the problems of motion jitter and instability in rigid collision contact caused by visual perception uncertainty in unstructured environments. The technical solution includes: First, constructing a semantic-geometric anchoring field by aligning a visual language model with CLIP features, transforming the uncertainty of visual positioning into the confidence radius of waypoints, and generating an energy-optimal B-spline baseline trajectory containing jerk minimization and visual relaxation constraints; Second, constructing a customized mother wavelet based on the robot's second-order underdamped mechanical characteristics, performing pulse adaptive continuous wavelet transform on the real-time current signal, and extracting weak contact features using matched filtering; Finally, inputting the temporal motion trend and frequency domain contact features into a dual-stream gated residual network, dynamically adjusting the fusion weights through an adaptive gating mechanism, and outputting the torque residual for compliant compensation. This invention has clear physical interpretability and significantly improves the robot's semantic understanding ability and contact stability in complex operations.

[0121] The terms “comprising” and “having”, and any variations thereof, in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, product, or apparatus that includes a series of steps or units, not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A compliant control method based on semantic anchoring and frequency domain pulse residuals, characterized in that, include: Acquire RGB images and text commands of the target operation scene, and construct a semantic-geometric anchoring field by aligning visual language models with CLIP features; Based on the constructed semantic-geometric anchoring field, the uncertainty of visual positioning is transformed into the confidence radius of waypoints, thereby generating an energy-optimal baseline trajectory that includes visual uncertainty constraints. Based on the real-time acquired robot joint state data, time-domain motion feature flow and frequency-domain contact feature flow are obtained. A customized wavelet transform algorithm based on the robot's mechanical damping characteristics is used to perform pulse adaptive frequency-domain feature extraction on the frequency-domain contact feature flow to obtain the contact feature time-frequency map. The time-domain motion feature stream and the contact feature time-frequency map are input into a dual-stream gated residual network to deduce and calculate the torque residual compensation value. The torque residual compensation value is then fused with the feedforward torque of the reference trajectory to drive the robot joint motors to execute.

2. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 1, characterized in that, Acquire RGB images and text commands of the target operation scene, and construct a semantic-geometric anchoring field using visual language models and CLIP feature alignment, including: Open vocabulary detection is performed on RGB images using a visual language model, and the bounding boxes of target objects are output. The CLIP model is used to calculate the feature similarity between text instructions and pixels within the bounding box, and an operation heatmap is generated. Based on the peak regions and dispersion of the heatmap, a set of spatial waypoints is output. and its corresponding confidence radius ,in With heat map at waypoints The variances at each location are positively correlated.

3. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 2, characterized in that, Generate the energy-optimal baseline trajectory that includes visual uncertainty constraints, including: Spatial waypoints are fitted using B-spline curves, and control points are solved by minimizing the energy loss function. ， in, For the planned joint trajectory. For accelerometer, it represents the impact energy of the system; For spatial waypoints, The confidence radius corresponding to the spatial waypoint; The balance coefficient is n; n is the number of spatial waypoints. To prevent tiny quantities with a denominator of zero; The control points of the B-spline curve to be optimized; the smoothness of the trajectory constrained by the first term of the energy loss function, and the geometric consistency between the trajectory constrained by the second term and the visual observations. The smaller the value, the better the trajectory. The stronger the anchoring effect.

4. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 1, characterized in that, The construction of the customized wavelet transform algorithm includes: The rigid collision process of the robot is modeled as the impulse response of a second-order underdamped system, and its time-domain expression is: ， in, The inherent damping ratio of the robot, The natural angular frequency, The frequency of the damped oscillation. It is the amplitude coefficient; based on Constructing a custom wavelet transform algorithm for: ， in, This is the term for removing the mean. is a Gaussian window function to ensure tight support in the time domain, and C is the energy normalization coefficient.

5. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 4, characterized in that, Based on the customized wavelet transform algorithm, the current signal... Perform continuous wavelet transform, the expression is: ， in, As a scale factor, The translation factor is... The output is a time-frequency diagram of the contact characteristics; this transformation process is equivalent to constructing a matched filter for the collision signal in the frequency domain.

6. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 1, characterized in that, The dual-stream gated residual network adopts a heterogeneous dual-stream structure, and its implementation process includes: Current joint position of the robot ,speed and reference trajectory error Input to the time-domain feature encoding module, output time-domain feature vector ; Contact feature time-frequency diagram Input frequency domain feature encoding module, output frequency domain contact feature vector ; The gating weight is calculated based on the energy intensity of the frequency domain contact feature vector. The dual-flow features are weighted and fused, and the torque residual compensation value is output based on the fusion result.

7. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 6, characterized in that, The control law calculation formula that integrates the torque residual compensation value and the feedforward torque of the reference trajectory is as follows: ， in, The command torque sent to the joint motor; This is the torque residual compensation value; The feedforward torque, calculated based on the dynamic model, is obtained from the generated baseline trajectory parameters. and PD control gain; As the reference joint angular velocity, This is the reference joint angular acceleration.

8. The compliant control method based on semantic anchoring and frequency domain pulse residuals according to claim 6, characterized in that, The parameters of the dual-stream gated residual network are optimized using a staged reinforcement learning strategy, including: In the first stage, the adaptive gating unit is frozen, the gating weight is set to 0, and the network is trained using only time-domain streaming data to enable it to have basic trajectory tracking capabilities. In the second stage, the adaptive gating unit is unfrozen, random rigid body collision interference is introduced into the simulation environment, and the network parameters are optimized using the near-end policy optimization algorithm. The reward function is designed as a weighted sum of the position tracking error term and the contact force control term, so that the network learns to adjust compliance using the torque residual compensation value at the moment of contact.

9. A compliant control system based on semantic anchoring and frequency domain pulse residuals, characterized in that, include: The perception planning unit is used to perform semantic-geometric anchoring and energy-optimal baseline trajectory generation, and outputs baseline trajectory parameters. The signal processing unit has a built-in custom wavelet transform algorithm for pulse adaptive feature extraction of current sensor signals and outputs a contact feature time-frequency diagram. The dual-stream inference unit loads a pre-trained dual-stream gated residual network to calculate torque residual compensation values ​​based on state data. The motion control execution unit is used to control the movement of the robot's joint motors by combining the feedforward torque and the torque residual compensation value.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to execute the compliant control method based on semantic anchoring and frequency domain pulse residuals as described in any one of claims 1 to 8.

Images