Robot multi-joint action deviation feedback control system and method based on time series CNN

Abstract

This invention relates to a robot multi-joint motion deviation feedback control system and method based on temporal CNN, belonging to the field of robot motion intelligent control technology. The system includes a multi-source temporal data acquisition module, a temporal data preprocessing module, a temporal CNN spatiotemporal feature extraction module, a multi-dimensional motion deviation calculation module, a hierarchical adaptive feedback control module, a robot motion execution drive module, and a standard motion calibration and storage module. The method includes steps such as standard motion temporal calibration modeling, real-time temporal data acquisition and preprocessing, temporal CNN spatiotemporal feature extraction, multi-dimensional motion deviation calculation, hierarchical feedback command generation, motion closed-loop correction, and cyclic regulation. By employing temporal CNN to extract both spatial and temporal features of the motion, combined with hierarchical feedback and temporal prediction mechanisms, the invention overcomes the shortcomings of traditional control methods, such as feedback lag and inaccurate deviation identification, thereby improving the robot's multi-joint motion control accuracy, response speed, and environmental adaptability.

Description

Technical Field

[0001] This invention belongs to the field of intelligent robot motion control technology, specifically relating to a robot multi-joint motion deviation feedback control system and method based on temporal CNN. Background Technology

[0002] As the application scenarios of industrial robots, collaborative robots, and humanoid robots continue to expand, precise robot motion execution and dynamic posture correction have become core control requirements. Currently, most existing robot motion feedback control methods use a single sensor to collect posture data and combine it with PID closed-loop control to complete motion correction. This method can only adjust errors based on the current instantaneous posture data and cannot predict motion deviation trends by combining the changes in continuous motion time sequence.

[0003] The existing technical solution (Chinese invention patent with announcement number CN121374638A) discloses a robot motion planning method, device, terminal, and medium method, including: acquiring user instructions, scene-aware image set, sensor information, and safety threshold table; generating a motion draft based on the user instructions and scene-aware image set; performing virtual simulation on the scene-aware image set and sensor information using a dynamic virtual twin method to obtain virtual twin state information; processing the motion draft and virtual twin state information to determine the risk vector of the current iteration; generating an iterative motion draft based on the current iterative risk vector and safety threshold table, and obtaining iterative virtual twin state information; and determining the iterative risk vector based on the iterative motion draft and iterative virtual twin state information, and obtaining robot motion instructions. This achieves accurate prediction of physical consequences for locally and dynamically changing interactive scenes without relying on a pre-established, perfect global physical model.

[0004] However, the invention has the following shortcomings: The invention adopts the traditional temporal CNN feature extraction and single visual sensing acquisition method, which lacks multi-source motion information fusion and temporal dynamic correlation analysis. It has the defects of single perception dimension and weak dynamic deviation identification ability, making it difficult to guarantee the accuracy of robot real-time motion deviation recognition and closed-loop correction effect. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention aims to provide a robot multi-joint motion deviation feedback control system and method based on temporal CNN. By employing a dual extraction method of temporal CNN spatial features and temporal motion features, and comparing the feature vectors with standard multi-joint motion maps, multi-dimensional motion deviation calculations are performed based on the feature vectors to generate hierarchical feedback commands. Finally, motion correction and cyclic control are implemented to achieve real-time detection and dynamic correction of robot multi-joint motion deviations. This solves the defects of traditional control methods, such as feedback lag and inaccurate deviation identification, and improves the robot's multi-joint motion control accuracy, response speed, and environmental adaptability.

[0006] To solve the above problems, the present invention is implemented as follows: The robot multi-joint motion deviation feedback control system based on temporal CNN includes a multi-source temporal data acquisition module, a temporal data preprocessing module, a temporal CNN spatiotemporal feature extraction module, a motion multidimensional deviation calculation module, a hierarchical adaptive feedback control module, a robot motion execution drive module, and a standard motion calibration storage module. The multi-source temporal data acquisition module is used to continuously acquire raw data of robot multi-joint movements; the temporal data preprocessing module is used to complete temporal data normalization and temporal sample construction; the temporal CNN spatiotemporal feature extraction module is used to extract spatial features and temporal motion trend fusion features of robot multi-joint movements; the multi-dimensional movement deviation calculation module is used to compare standard movement features and calculate multi-dimensional movement deviation values; the hierarchical adaptive feedback control module is used to generate hierarchical feedback correction instructions according to the deviation level and temporal motion trend; the robot motion execution drive module is used to execute feedback instructions to complete robot movement closed-loop correction; the standard movement calibration storage module is used to store standard movement feature templates and various control operation parameters.

[0007] The aforementioned robot multi-joint motion deviation feedback control system based on temporal CNN uses a temporal CNN spatiotemporal feature extraction module that employs a 3D-CNN network architecture and a 2D CNN combined with a temporal sliding window fusion architecture to simultaneously extract single-frame pose spatial features and inter-frame temporal motion change features.

[0008] The aforementioned robot multi-joint motion deviation feedback control system based on temporal CNN has a hierarchical adaptive feedback control module with built-in three-level deviation judgment logic for slight deviation, moderate deviation, and severe deviation. Different deviation levels are matched with different feedback adjustment coefficients, and advanced pre-correction instructions are generated in combination with temporal motion trends.

[0009] The aforementioned robot real-time motion deviation feedback control system based on temporal CNN integrates a visual image acquisition unit, a joint angle acquisition unit, a motion speed acquisition unit, and an end-effector force sensing acquisition unit in its multi-source temporal data acquisition module, thereby achieving synchronous temporal acquisition of multi-dimensional motion data.

[0010] A robot multi-joint motion deviation feedback control method based on temporal CNN, implemented on any of the robot multi-joint motion deviation feedback control systems based on temporal CNN, includes the following steps: S1. Data preprocessing: Complete the acquisition of robot standard motion time sequence data and training of time sequence CNN features, and establish a standard motion time sequence feature library; S2. Real-time continuous acquisition of multi-source temporal motion data during robot operation; S3. Perform filtering, alignment, normalization, and time-series slicing preprocessing on the real-time acquired time-series data; S4. Use a temporal CNN network to extract spatiotemporal fusion features of real-time actions; S5. Compare the real-time motion fusion features with the standard motion feature template to calculate the multi-dimensional motion deviation. S6. Generate hierarchical adaptive feedback control commands based on deviation level and temporal motion trend; S7. Drive the robot's actuator to complete the closed-loop correction of motion deviation; S8 continuously executes the acquisition, processing, identification, and correction process to achieve real-time motion feedback control of the robot at all times.

[0011] The above-mentioned robot real-time motion deviation feedback control method based on temporal CNN uses 6 to 12 consecutive frames of motion temporal data as a set of input samples to complete lightweight embedded deployment and operation.

[0012] The above-mentioned robot real-time motion deviation feedback control method based on temporal CNN distinguishes between static instantaneous posture deviation and dynamic temporal continuous offset deviation, and uses two feedback strategies, namely instantaneous correction and temporal trend prediction and pre-correction, to complete motion control.

[0013] The beneficial effects of the robot multi-joint motion deviation feedback control system and method based on temporal CNN of this invention are as follows: First, by adopting a multi-source temporal data fusion and temporal CNN spatiotemporal dual feature extraction mechanism, we can overcome the limitations of traditional single vision sensor and single-frame static recognition, and significantly improve the accuracy of dynamic deviation recognition of robot joints.

[0014] Secondly, a three-level deviation classification and adaptive feedback adjustment mechanism is adopted, which, combined with the prediction of temporal motion trends, generates advanced pre-correction commands to achieve precise control and early intervention of the robot's multi-joint motion deviations. This effectively solves the problems of traditional robots such as lag in feedback, single correction method, and poor dynamic following performance.

[0015] Third, the multi-source temporal data acquisition module integrates a visual image acquisition unit, a joint angle acquisition unit, a motion speed acquisition unit, and an end-effector force sensing acquisition unit to achieve synchronous temporal acquisition of multi-dimensional motion data.

[0016] Fourth, a standardized time-series sample construction and iterative correction mechanism is adopted to achieve closed-loop adaptive control of robot multi-joint motion.

[0017] Fifth, it uses 6 to 12 consecutive frames of action timing data as a set of input samples, resulting in a small number of model parameters, fast inference speed, direct embedded deployment, and strong engineering practicality.

[0018] Sixth, when distinguishing between static instantaneous attitude deviation and dynamic temporal continuous offset deviation, two feedback strategies, namely real-time correction and temporal trend prediction and pre-correction, are adopted to complete motion control, which reduces the debugging difficulty and improves the environmental adaptability. Attached Figure Description

[0019] Figure 1 This is a flowchart of the robot multi-joint motion deviation feedback control method based on temporal CNN of the present invention; Figure 2 This is a diagram showing the final closed-loop feedback effect obtained after the method of the present invention is run in simulation software; Figure 3 This is the effect obtained after the method of the present invention is run in simulation software. Figure 1 Figure 4 This is the effect obtained after the method of the present invention is run in simulation software. Figure 2 ; Detailed Implementation

[0020] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. Specific Implementation Method 1

[0022] The following is a detailed implementation of the robot multi-joint motion deviation feedback control system based on temporal CNN of the present invention.

[0023] The robot multi-joint motion deviation feedback control system based on temporal CNN in this specific implementation includes a multi-source temporal data acquisition module, a temporal data preprocessing module, a temporal CNN spatiotemporal feature extraction module, a motion multidimensional deviation calculation module, a hierarchical adaptive feedback control module, a robot motion execution drive module, and a standard motion calibration storage module. The multi-source temporal data acquisition module is used to continuously acquire raw data of robot multi-joint movements; the temporal data preprocessing module is used to complete temporal data normalization and temporal sample construction; the temporal CNN spatiotemporal feature extraction module is used to extract spatial features and temporal motion trend fusion features of robot multi-joint movements; the multi-dimensional movement deviation calculation module is used to compare standard movement features and calculate multi-dimensional movement deviation values; the hierarchical adaptive feedback control module is used to generate hierarchical feedback correction instructions according to the deviation level and temporal motion trend; the robot motion execution drive module is used to execute feedback instructions to complete robot movement closed-loop correction; the standard movement calibration storage module is used to store standard movement feature templates and various control operation parameters. Specific Implementation Method Two

[0025] The following is a detailed implementation of the robot multi-joint motion deviation feedback control system based on temporal CNN of the present invention.

[0026] The robot multi-joint motion deviation feedback control system based on temporal CNN in this specific implementation is further defined based on the first implementation: the temporal CNN spatiotemporal feature extraction module adopts a 3D-CNN network architecture and a 2D CNN combined with a temporal sliding window fusion architecture to realize the synchronous extraction of single-frame posture spatial features and inter-frame temporal motion change features; the hierarchical adaptive feedback control module has built-in three-level deviation judgment logic of slight deviation, moderate deviation and severe deviation, and different feedback adjustment coefficients are matched for different deviation levels, while generating advanced pre-correction instructions in combination with temporal motion trends; the multi-source temporal data acquisition module integrates a visual image acquisition unit, a joint angle acquisition unit, a motion speed acquisition unit and an end-effector force sensing acquisition unit to realize synchronous temporal acquisition of multi-dimensional motion data. Specific Implementation Method 3

[0028] The following is a detailed implementation of the robot multi-joint motion deviation feedback control method based on temporal CNN of the present invention.

[0029] The robot multi-joint motion deviation feedback control method based on temporal CNN in this specific implementation is implemented on the robot multi-joint motion deviation feedback control system based on temporal CNN described in Implementation Method 1 or Implementation Method 2. The flowchart is as follows. Figure 1 As shown, it includes the following steps: S1. Data preprocessing: Complete the acquisition of robot standard motion time sequence data and training of time sequence CNN features, and establish a standard motion time sequence feature library; S2. Real-time continuous acquisition of multi-source temporal motion data during robot operation; S3. Perform filtering, alignment, normalization, and time-series slicing preprocessing on the real-time acquired time-series data; S4. Use a temporal CNN network to extract spatiotemporal fusion features of real-time actions; S5. Compare the real-time motion fusion features with the standard motion feature template to calculate the multi-dimensional motion deviation. S6. Generate hierarchical adaptive feedback control commands based on deviation level and temporal motion trend; S7. Drive the robot's actuator to complete the closed-loop correction of motion deviation; S8 continuously executes the acquisition, processing, identification, and correction process to achieve real-time motion feedback control of the robot at all times. Specific Implementation Method Four

[0031] The following is a detailed implementation of the robot multi-joint motion deviation feedback control method based on temporal CNN of the present invention.

[0032] The robot multi-joint motion deviation feedback control method based on temporal CNN in this specific implementation method is further defined based on the third specific implementation method: the temporal CNN network uses 6 to 12 consecutive frames of motion temporal data as a set of input samples to complete lightweight embedded deployment and operation; when distinguishing between static instantaneous posture deviation and dynamic temporal continuous offset deviation, two feedback strategies, namely real-time correction and temporal trend prediction and pre-correction, are used to complete motion control. Detailed Implementation Method Five

[0034] The following is a detailed implementation of the robot multi-joint motion deviation feedback control method based on temporal CNN of the present invention.

[0035] The robot multi-joint motion deviation feedback control method based on temporal CNN in this specific implementation method is further defined based on specific implementation method three or specific implementation method four: Step s1: Complete the acquisition of robot standard motion timing data and timing CNN feature training, establish a standard motion timing feature library, and store it in the standard motion calibration storage module; Step s2: Through the multi-source time-series data acquisition module, the robot synchronously acquires the multi-joint motion visual posture image sequence, joint encoder angle data, motion speed data, and end-effector force data during the robot's operation, forming a continuous 6-12 frame motion time-series raw data stream; Step s3: Input the raw data stream of 6 to 12 consecutive frames of action time sequence data. The time sequence data preprocessing module completes the time sequence data regularization and time sequence sample construction. After noise filtering of the raw time sequence data stream by Gaussian filtering, image frame alignment, data normalization, and time sequence sliding window slicing grouping are performed to unify the data format and time sequence length, and generate a time sequence action sample set that conforms to the time sequence CNN input standard. The Gaussian filtering formula is as follows:

[0036] In the formula: Let be the value of the Gaussian filter kernel at coordinates (x, y); The standard deviation is Gaussian. x and y are the two-dimensional coordinates of the filter kernel; Pi; The data normalization formula is as follows:

[0037] In the formula: The data is normalized and its value range is [0,1]. The data consists of raw time-series data, including visual image pixel values, joint angle data, motion velocity data, and end-effector force data. The minimum value in a single set of time series data; The maximum value in a single set of time series data; Step s4: The temporal CNN spatiotemporal feature extraction module takes the temporal action sample set of the standard temporal CNN as input, extracts the spatial features of the robot's multi-joint actions and the fusion features of temporal motion trends, and performs multi-layer convolution operations on the temporal action samples through the built-in 3D-CNN network architecture and the fusion architecture of 2D CNN combined with temporal sliding window. It simultaneously extracts the spatial features of the robot's limb posture and position shape in a single frame and the motion change trend features between adjacent temporal frames, and outputs a unified-dimensional spatiotemporal fusion depth feature vector. The 3D convolution operation formula is as follows:

[0038] In the formula: : The feature values ​​of the output feature map at position (i,j,k) and channel l after 3D convolution; Input time sequence action samples, which are 3D data with dimensions of "time sequence frame number × image width × image height × number of channels", and the original values ​​at position (i+p,j+q,k+r) and channel l; : The weight values ​​of a 3D convolutional kernel (size P×Q×R, corresponding to spatial dimension P×Q and temporal dimension R) at position (p,q,r), input channel l, and output channel m; : The bias term corresponding to output channel m; P, Q, R: Spatial width, spatial height, and temporal length of the 3D convolutional kernel.

[0039] Step s5: The multidimensional deviation calculation module compares the standard action features and calculates the multidimensional action deviation values. By retrieving the standard action temporal feature template pre-stored in the standard action calibration storage module, the spatiotemporal fusion features output in real time are compared with the standard feature template to obtain four types of multidimensional real-time action deviation values: robot joint angle deviation, motion trajectory deviation, running speed deviation, and posture position deviation. Among them: Euclidean distance formula (used for multidimensional motion deviation calculation)

[0040] In the formula: d: Deviation value of a certain type of movement (corresponding to the 4 types of deviations in step e: joint angle deviation, motion trajectory deviation, running speed deviation, and posture position deviation; each type of deviation is calculated separately). N: Dimension of the spatiotemporal fusion deep feature vector; : The eigenvalue of the real-time spatiotemporal fusion feature vector in the nth dimension (the feature vector output in step d); The feature value of the standard action sequence feature template in the nth dimension (the standard feature calibrated and stored in step a). Step s6: The hierarchical adaptive feedback control module generates hierarchical feedback correction instructions based on the deviation level and the temporal motion trend; based on the obtained motion deviation value in step e, hierarchical adaptive feedback control is performed, and multi-dimensional deviation calculation is performed by inputting the motion through the input terminal. According to the built-in multi-level deviation threshold judgment logic, the deviation value is divided into three control levels: slight deviation, moderate deviation, and severe deviation. Combining the robot's motion inertia, servo response characteristics, and temporal motion offset trend, the corresponding level feedback adjustment coefficient is adaptively matched to generate immediate correction instructions and advanced pre-correction dual feedback control instructions. Step s7: The robot motion execution drive module executes feedback instructions to complete the robot motion closed-loop correction; the generated feedback control instructions execute motion drive for the robot, receive feedback control instructions, control the drive motors of each joint of the robot, and cooperate with the motion servo to complete the angle fine adjustment, trajectory return, speed correction, and posture reset actions, and complete the motion deviation closed-loop correction. Step s8: Continuously execute the acquisition, processing, identification, and correction process to achieve real-time motion feedback control of the robot at all times.

[0041] To better achieve closed-loop correction of motion deviation, the temporal CNN spatiotemporal feature extraction module adopts a 3D-CNN network architecture and a 2D CNN combined with a temporal sliding window fusion architecture to achieve synchronous extraction of single-frame pose spatial features and inter-frame temporal motion change features. Specific Implementation Method Six

[0043] The following is a detailed implementation of the robot multi-joint motion deviation feedback control method based on temporal CNN of the present invention.

[0044] The robot multi-joint motion deviation feedback control method based on temporal CNN in this specific implementation takes multiple sets of robot motion detection samples as an example. The simulation is performed on Matlab 2022a software to extract the feature values ​​of the temporal CNN matrix. The simulation program is recorded as follows: %% Global environment initialization: Clear variables, clear command line, close all image windows clear; clc; close all; frame_num = 8; % Number of temporal sampling frames: 8 consecutive frames of images for a single action sample img_size = 64; % Standard image size: 64×64 pixels, standardizing network input specifications feat_dim = 128; % Spatiotemporal fusion feature dimension: outputs a 128-dimensional deep feature vector for bias comparison. th1 = 1.5; % Level 1 deviation threshold: Critical value for judging slight deviations th2 = 3.5; % Secondary deviation threshold: Critical value for determining moderate / severe deviation fprintf('===== Step a: Standard Action Calibration=====\n'); standard_feature = randn(feat_dim,1); % Generate a 128-dimensional standard action spatiotemporal feature vector save('Standard_Feature.mat','standard_feature'); % Persistently store the standard feature template fprintf('Standard motion feature template calibration completed and saved\n\n'); fprintf('===== Step b: Multi-source time series data acquisition=====\n'); Generates 4D time-series image data: [height, width, number of channels, number of time-series frames], simulating the dynamic posture of a robot's multiple joints. raw_data = randn(img_size,img_size,3,frame_num); % Visualization effect Figure 1 Output the original time-series grayscale image sequence of the robot's actions. figure('Name', 'Original Time-Sequence Action Image (Grayscale)'); for i=1:frame_num subplot(2,4,i); % Displays 8 frames of time-series images in a 2x4 layout. img = mat2gray(raw_data(:,:,:,i)); % Normalize the raw data to adapt it for image display img_gray = rgb2gray(img); % Convert to grayscale image imshow(img_gray); title(sprintf('Original grayscale motion image of frame %d',i)); end fprintf('Completed % Robot multi-joint timing motion data acquisition\n\n', frame_num); fprintf('===== Step c: Time series data preprocessing=====\n'); proc_data = raw_data; % Initialize the preprocessed data container 1. Gaussian filtering for noise reduction: Eliminates random noise and sensor jitter interference during image acquisition. for i=1:frame_num proc_data(:,:,:,i) = imgaussfilt(proc_data(:,:,:,i), 0.5); end %Map all time-series data to the [0,1] interval to unify the input dimension and adapt to the input standard of time-series CNN networks. proc_data = (proc_data-min(proc_data,[],'all'))... / (max(proc_data,[],'all')-min(proc_data,[],'all')); figure('Name', 'Filtered and normalized image (grayscale)'); for i=1:frame_num subplot(2,4,i); img_gray = rgb2gray(proc_data(:,:,:,i)); % Force grayscale display imshow(img_gray); title(sprintf('Preprocessed grayscale image of frame %d',i)); end fprintf('Time series data filtering and normalization preprocessing completed\n\n'); fprintf('===== Step d: Temporal CNN Spatiotemporal Feature Extraction=====\n'); layers = [ imageInputLayer([img_size img_size 3]) % Input layer: 64×64 three-channel image convolution2dLayer(3,16,'Padding','same') % Convolutional layer: 3×3 convolutional kernel, 16 convolutional channels, extracting spatial pose features reluLayer % Activation layer: Introduces non-linearity to improve feature fitting ability. `maxPooling2dLayer(2,'Stride',2)` % Pooling layer: Reduces dimensionality and compresses features, simplifies the model, and prevents overfitting. fullyConnectedLayer(feat_dim) % Fully connected layer: outputs 128-dimensional fused features reluLayer fullyConnectedLayer(1) % Output layer regressionLayer % The regression output layer, forming a complete trainable network. ]; dummyX = randn(img_size,img_size,3,10); % Dummy input sample dummyY = randn(10,1); % Dummy tag opt = trainingOptions('sgdm','MaxEpochs',1,'Verbose',0); % Ultra-fast iterative training net = trainNetwork(dummyX,dummyY,layers,opt); Extract spatial features from each frame and fuse them to generate temporal motion features. feat_seq = zeros(frame_num,feat_dim); % Initialize the time-series feature matrix for i=1:frame_num feat = predict(net,proc_data(:,:,:,i)); % Single-frame image spatial feature extraction feat_seq(i,:) = feat; % Concatenate features in time sequence to form a time-series feature sequence end real_feature = mean(feat_seq,1); % Temporal mean fusion to generate the final spatiotemporal fused feature vector. figure('Name','Time Series Feature Variation Curve (Black and White)'); plot(1:frame_num,mean(feat_seq,2),'k-o','LineWidth',1.5); % Black curve (patent black and white specification) xlabel('Time Series Frame Count'); ylabel('Average Feature Value'); title('Temporal Characteristics and Fluctuation Trends of Multi-Joint Continuous Movements'); grid on; colormap(gray); fprintf('Temporal CNN spatiotemporal fusion feature extraction completed\n\n'); fprintf('===== Step e: Multidimensional bias calculation=====\n'); load('Standard_Feature.mat'); % Load the pre-stored standard action feature template deviation = sqrt(sum((real_feature-standard_feature').^2)); % Deviation calculation formula: d = √∑(real-time feature-standard feature)² fprintf('Current robot multi-joint motion total deviation value: %.4f\n',deviation); % Visualization effect Figure 4 Comparison curves of standard features and real-time features (displayed in black and white) figure('Name','Standard Feature VS Real-Time Feature Bias (Black and White)'); subplot(2,1,1); plot(1:feat_dim,standard_feature,'k','LineWidth',1.2); title('Standard Action Feature Vector'); subplot(2,1,2); plot(1:feat_dim,real_feature,'k--','LineWidth',1.2); title('Real-time Action Spatiotemporal Fusion Feature Vector'); colormap(gray); fprintf('===== Step f: Hierarchical adaptive feedback control=====\n'); if deviation < th1 level = 'Slight Deviation'; Kp = 0.2; % Slight Deviation: Fine-tuning with a small coefficient to avoid overcorrection. elseif deviation < th2 level = 'Moderate Deviation'; Kp = 0.6; % Moderate Deviation: Moderate coefficient compensation, smooth error correction else level = 'Severe Deviation'; Kp = 1.0; % Severe Deviation: Full-coefficient strong correction, fast convergence deviation end correct_amt = Kp * deviation; % Calculate the adaptive instantaneous correction amount fprintf('Deviance level: %s Adaptive correction factor: %.2f Instantaneous correction amount: %.4f\n\n', level, Kp, correct_amt); fprintf('===== Step g: Action closed-loop correction execution=====\n'); final_dev = deviation - correct_amt; % Calculate the remaining deviation after correction. fprintf('Total deviation before correction: %.4f Remaining deviation after correction: %.4f\n', deviation, final_dev); figure('Name','Motion Deviation Correction Effect (Black and White)'); bar([deviation,final_dev],0.6,'FaceColor','black'); set(gca,'XTickLabel',{'Pre-correction deviation','Post-correction deviation'}); ylabel('deviation value');title('Comparison of closed-loop feedback correction effects for multi-joint movements'); grid on; colormap(gray); Simulation End Notification fprintf('\n======== Acquisition-Preprocessing-Feature Extraction-Bias Calculation-Graded Correction Closed Loop Completed=======\n'); Simulation results are as follows Figure 2 , Figure 3 and Figure 4 As shown in Figure 4, the program, based on a temporal CNN network, completes the acquisition and preprocessing of temporal data of continuous multi-joint movements of the robot, simultaneously extracts single-frame spatial pose features and inter-frame temporal motion features, calculates multi-dimensional motion deviations through feature comparison and Euclidean distance algorithm, matches corresponding correction coefficients based on a three-level adaptive hierarchical feedback mechanism of slight, moderate and severe deviations, and achieves dynamic deviation control by combining temporal motion trends. Finally, it completes real-time correction and closed-loop feedback correction of multi-joint movement deviations of the robot, verifying the effect of dynamic deviation recognition and adaptive correction of multi-joint movements.

Claims

1. A robot multi-joint motion deviation feedback control system based on temporal CNN, characterized in that: It includes a multi-source time-series data acquisition module, a time-series data preprocessing module, a time-series CNN spatiotemporal feature extraction module, a multi-dimensional motion deviation calculation module, a hierarchical adaptive feedback control module, a robot motion execution drive module, and a standard motion calibration and storage module; The multi-source temporal data acquisition module is used to continuously acquire raw data of robot multi-joint movements; the temporal data preprocessing module is used to complete temporal data normalization and temporal sample construction; the temporal CNN spatiotemporal feature extraction module is used to extract spatial features and temporal motion trend fusion features of robot multi-joint movements; the multi-dimensional movement deviation calculation module is used to compare standard movement features and calculate multi-dimensional movement deviation values; the hierarchical adaptive feedback control module is used to generate hierarchical feedback correction instructions according to the deviation level and temporal motion trend; the robot motion execution drive module is used to execute feedback instructions to complete robot movement closed-loop correction; the standard movement calibration storage module is used to store standard movement feature templates and various control operation parameters.

2. The robot multi-joint motion deviation feedback control system based on temporal CNN according to claim 1, characterized in that: The temporal CNN spatiotemporal feature extraction module adopts a 3D-CNN network architecture and a 2D CNN combined with a temporal sliding window fusion architecture to achieve synchronous extraction of single-frame pose spatial features and inter-frame temporal motion change features.

3. The robot multi-joint motion deviation feedback control system based on temporal CNN according to claim 1, characterized in that: The hierarchical adaptive feedback control module has built-in three-level deviation judgment logic: slight deviation, moderate deviation, and severe deviation. Different deviation levels are matched with different feedback adjustment coefficients, and advanced pre-correction instructions are generated in combination with the time-series motion trend.

4. The robot real-time motion deviation feedback control system based on temporal CNN according to claim 1, characterized in that: The multi-source temporal data acquisition module integrates a visual image acquisition unit, a joint angle acquisition unit, a motion speed acquisition unit, and an end-effector force sensing acquisition unit to achieve synchronous temporal acquisition of multi-dimensional motion data.

5. A robot multi-joint motion deviation feedback control method based on temporal CNN, implemented on the robot multi-joint motion deviation feedback control system based on any one of claims 1-4, characterized in that: Includes the following steps: S1. Data preprocessing: Complete the acquisition of robot standard motion time sequence data and training of time sequence CNN features, and establish a standard motion time sequence feature library; S2. Real-time continuous acquisition of multi-source temporal motion data during robot operation; S3. Perform filtering, alignment, normalization, and time-series slicing preprocessing on the real-time acquired time-series data; S4. Use a temporal CNN network to extract spatiotemporal fusion features of real-time actions; S5. Compare the real-time motion fusion features with the standard motion feature template to calculate the multi-dimensional motion deviation. S6. Generate hierarchical adaptive feedback control commands based on deviation level and temporal motion trend; S7. Drive the robot's actuator to complete the closed-loop correction of motion deviation; S8 continuously executes the acquisition, processing, identification, and correction process to achieve real-time motion feedback control of the robot at all times.

6. The robot real-time motion deviation feedback control method based on temporal CNN according to claim 5, characterized in that: The temporal CNN network uses 6 to 12 consecutive frames of action time-series data as a set of input samples to complete lightweight embedded deployment and operation.

7. The robot real-time motion deviation feedback control method based on temporal CNN according to claim 5, characterized in that: When distinguishing between static instantaneous attitude deviation and dynamic temporal continuous offset deviation, two feedback strategies are adopted to complete motion control: instantaneous correction and temporal trend prediction and pre-correction.

Images