A robot testing method and system based on big data and artificial intelligence
By employing multimodal data acquisition and fusion technology, combined with improved deep neural networks and reinforcement learning algorithms, the shortcomings of traditional robot testing methods in terms of coverage and accuracy have been addressed, enabling efficient and intelligent robot testing.
Patent Information
- Application Number
- CN202511417495.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-30
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2045-09-30
AI Technical Summary
Traditional robot testing methods suffer from limited test coverage, poor adaptability, insufficient evaluation accuracy, and difficulty in handling multimodal data. Existing big data and artificial intelligence-based methods suffer from insufficient model generalization ability and poor multimodal data fusion performance.
Adaptive test cases are constructed by employing multimodal data acquisition, improved deep neural network models, reinforcement learning algorithms, and dynamic weight fusion technology. Visual and motion data are processed through improved convolutional neural networks and recurrent neural networks, multimodal features are fused by combining attention mechanisms, and adaptive test cases are generated through reinforcement learning and dynamic weight fusion algorithms are used to optimize test results.
It improves the comprehensiveness, accuracy, and efficiency of testing, enhances the ability to extract complex features, significantly improves test coverage and the reliability of evaluation results, and provides intelligent performance optimization suggestions.
Smart Images
Figure CN120901980B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of robot testing technology, and more specifically, to a robot testing method and system based on big data and artificial intelligence. Background Technology
[0002] With the rapid development of robotics technology, the application scenarios of robots are becoming increasingly widespread, from industrial production to home services, from medical care to education and training; robots are gradually penetrating all aspects of social life. The performance and safety of robots directly affect their application effectiveness and user experience; therefore, comprehensive, efficient, and accurate testing of robots is crucial.
[0003] Traditional robot testing methods primarily rely on manually designed test cases to verify the robot's various functions by simulating specific scenarios. This method has the following shortcomings:
[0004] Limited test coverage: Manually designed test cases are difficult to cover all possible scenarios and are prone to missing potential problems.
[0005] Poor adaptability: When the robot's function or application scenario changes, a large number of test cases need to be redesigned, which is costly and inefficient.
[0006] Insufficient accuracy in assessment: Traditional testing methods rely heavily on human judgment, which is highly subjective and prone to misjudgment.
[0007] Difficulty in handling multimodal data: Modern robots are equipped with multiple sensors, and the resulting multimodal data is difficult to analyze effectively using traditional methods.
[0008] In recent years, the development of big data and artificial intelligence technologies has provided new ideas for robot testing. Some studies have attempted to apply machine learning algorithms to robot testing, but existing methods still suffer from problems such as insufficient model generalization ability, inflexible testing strategies, and poor multimodal data fusion.
[0009] Therefore, a robot testing method and system based on big data and artificial intelligence is proposed to address the above problems. Summary of the Invention
[0010] To overcome the aforementioned deficiencies of the prior art, this application provides a robot testing method and system based on big data and artificial intelligence to solve the problems mentioned in the background art.
[0011] To achieve the above objectives, this application provides the following technical solution: a robot testing method based on big data and artificial intelligence, the testing method comprising the following steps:
[0012] S1. Construct a multimodal test data acquisition framework to collect multimodal data of the robot in real time during the testing process;
[0013] S2. Preprocess the collected multimodal data, including data cleaning, data standardization, and feature extraction;
[0014] S3. Construct an improved deep neural network model to evaluate the robot's functions and performance;
[0015] S4. Generate adaptive test cases based on reinforcement learning algorithms to optimize test coverage;
[0016] S5. The output results of the improved deep neural network model are integrated using a dynamic weight fusion algorithm to obtain the final test conclusion;
[0017] S6. Generate robot performance optimization suggestions based on the test results.
[0018] Furthermore, the multimodal data in S1 includes visual data, motion data, environmental data, and system status data. The collected multimodal data is synchronized via timestamps and stored in a distributed data storage system. The data organization structure used for storing the multimodal data is as follows: ,in For multimodal datasets, For timestamps, For visual data, For motion data, For environmental data, For system status data, The total number of samples.
[0019] Furthermore, the S2 data cleaning uses an improved Z-score algorithm to detect outliers, and missing values are filled using an LSTM-based time series prediction model.
[0020] In data standardization, Z-score standardization is used for data that follows a Gaussian distribution, min-max standardization is used for data whose values are within a finite interval, and pixel values are normalized to the [0,1] interval for image data.
[0021] Feature extraction employs an improved autoencoder for data denoising and feature extraction. The improved autoencoder includes an encoder and a decoder, with the encoder incorporating the input data. Mapping to a low-dimensional feature space yields feature vectors. The decoder will use feature vectors Reconstructing the original data space yields reconstructed data. .
[0022] Furthermore, the improved deep neural network model in S3 includes:
[0023] An improved convolutional neural network for processing visual sensor data in robots;
[0024] An improved recurrent neural network for processing temporal motion data of robots;
[0025] The attention mechanism fusion module is used to integrate features from different modalities.
[0026] Among them, the improved convolutional neural network, namely MC-CNN, is designed for robot vision data processing. It introduces deformable convolutional kernels to capture irregular features. The specific strategy is as follows: the output feature mapping calculation formula for deformable convolution is: ,in To output feature mapping at position The value; The kernel size; For the first Each convolutional weight; For input feature mapping; , The learnable offset parameters are predicted through additional convolutional layers: ,in Predict the weights of the convolutional layer for the offset. As the activation function, the tanh function is used to limit the offset to a certain value. Within the range.
[0027] The network structure of MC-CNN is as follows:
[0028] Input layer: Receives 224×224×3 RGB images or depth images;
[0029] Deformable convolution block 1: 32 3×3 deformable convolution kernels, stride 1, padding 1, followed by BatchNorm and ReLU;
[0030] Pooling layer 1: 2×2 max pooling, step size 2;
[0031] Deformable convolution block 2: 64 3×3 deformable convolution kernels, stride 1, padding 1, followed by BatchNorm and ReLU;
[0032] Pooling layer 2: 2×2 max pooling, step size 2;
[0033] Deformable convolution block 3: 128 3×3 deformable convolution kernels, stride 1, padding 1, followed by BatchNorm and ReLU;
[0034] Global average pooling layer: converts the feature map into a 128-dimensional feature vector;
[0035] Output layer: Set the corresponding output dimensions and activation functions according to the specific task (such as object recognition, defect detection).
[0036] Furthermore, the improved recurrent neural network, namely MB-GRU, employs a bidirectional gated recurrent unit (GRU) to capture the temporal dependencies between the past and future, specifically using the following strategy:
[0037] MB-GRU consists of a forward GRU and a backward GRU. The forward GRU processes data from the beginning to the end of the sequence, while the backward GRU processes data from the end to the beginning of the sequence. Finally, the hidden states from both directions are concatenated as the output.
[0038] The update formula for a single GRU unit is:
[0039] Whether to update the hidden state of the door control is as follows: ;
[0040] Reset whether the door control ignores historical hidden states, specifically: ;
[0041] The candidate hidden state is as follows: ;
[0042] The final hidden state is as follows: ;
[0043] in, To update the door; To reset the door; Currently hidden; It is the sigmoid activation function; This is element-wise multiplication; , , Enter them separately The weight matrix that acts on the update gate, reset gate, and candidate hidden states; , These are the historical hidden states. The weight matrix applied to the update gate and the reset gate; Historical state after resetting the gate filter Acts on candidate hidden state The weight matrix; , , These are the bias vectors for the update gate, reset gate, and candidate hidden state, respectively.
[0044] The output of the bidirectional GRU is:
[0045] Forward GRU hidden state ;
[0046] Backward GRU hidden state ;
[0047] Concatenate forward and backward hidden states: ;
[0048] in This is the previous forward hidden state; This is the backward hidden state at the next moment.
[0049] The network structure of MB-GRU is as follows:
[0050] Input layer: Receives a sequence of temporal feature vectors of dimension d.
[0051] Bidirectional GRU layer 1: With 64 hidden units, it returns the complete sequence.
[0052] Bidirectional GRU layer 2: with 32 hidden units, returns the complete sequence.
[0053] Temporal attention layer: Assigns attention weights to different time steps in the sequence.
[0054] Fully connected layer: Maps the output to the corresponding dimension for evaluation of robot motion stability, trajectory accuracy, etc.
[0055] Furthermore, the attention mechanism fusion module, namely AMFM, integrates features from different modalities to learn the weights of features from different modalities, and the specific strategy of the attention mechanism fusion module is as follows:
[0056] Suppose that after feature extraction, we get Feature vectors of each modality ,and ,in Indicates the first Feature vectors of each modality; The dimension of the feature vector. express A real vector of dimension;
[0057] The attention mechanism fusion process is as follows:
[0058] Feature mapping: Mapping modal features of different dimensions to the same dimensional space. ,in and For learnable parameters, For a unified feature dimension; For the first The output feature vector after mapping of each modal feature; For the first The original feature vectors of each modality; This is the weight matrix; For one A bias vector of dimension; express A dimensional real vector;
[0059] Attention weight calculation: Weights are calculated based on the similarity between features and context vectors, specifically as follows: ,in For the first Attention weights for each modality feature; These are learnable context vectors; The cosine similarity function;
[0060] Feature fusion: Weighted summation yields the final fused features. Specifically: ;
[0061] Feature enhancement: Enhancing fused features through residual connections and nonlinear transformations Specifically: ,in, This is a residual connectivity feature; and These are learnable parameters.
[0062] Furthermore, the specific strategy for generating adaptive test cases using the reinforcement learning algorithm in S4 is as follows:
[0063] Modeling the reinforcement learning environment: Define the state space. This is represented as a combination of the robot's current state and the test environment state, specifically: ,in , , and These are feature extraction functions for visual, motion, environmental, and system state data, respectively. This is the current test coverage vector; , , and The proportion is Real-time visual, motion, environmental, and system state data; motion space A set of executable test operations. ,in For the first Each test action can be an operation, such as changing ambient lighting, adding distractions, or adjusting task difficulty. For the continuous action space, a Gaussian distribution is used to represent the action probability distribution.
[0064] reward function Configure settings to improve test coverage, determine the severity of issues discovered, and optimize test efficiency, specifically as follows: ,in The amount by which test coverage is increased; A collection of newly discovered problems; For the question The severity; Test duration; , , These are the weighting coefficients, and ; To perform the action The state after;
[0065] The reinforcement learning algorithm in S4 adopts an improved deep deterministic policy gradient algorithm, namely M-DDPG, which introduces priority experience replay and adaptive learning mechanisms.
[0066] Furthermore, the dynamic weights in the dynamic weight fusion algorithm used in S5 are adaptively adjusted according to the performance of each model.
[0067] Furthermore, the performance optimization suggestions in S6 are based on the test evaluation results and the constructed test knowledge graph, providing specific suggestions for the optimization of the robot's software and hardware.
[0068] Furthermore, a robot testing system based on big data and artificial intelligence includes: a multimodal data acquisition module for collecting various sensor data and operational status data of the robot;
[0069] The data preprocessing module is used to clean, standardize, and extract features from the collected data;
[0070] The deep learning evaluation module contains several improved neural network models for evaluating various aspects of the robot's performance.
[0071] The intelligent test case generation module dynamically generates test cases based on reinforcement learning algorithms.
[0072] The results fusion module uses a dynamic weight fusion algorithm to integrate the evaluation results of multiple models;
[0073] The feedback optimization module generates robot performance optimization suggestions based on the test results.
[0074] The multimodal data acquisition module includes:
[0075] The visual data acquisition unit includes a high-definition camera and a depth sensor for acquiring visual data;
[0076] The motion data acquisition unit includes an accelerometer, a gyroscope, and a force sensor for acquiring motion data;
[0077] The environmental data acquisition unit includes temperature and humidity sensors, air pressure sensors, and sound sensors for collecting environmental data;
[0078] The system status acquisition unit is used to collect data on the robot's CPU usage, memory consumption, and battery status.
[0079] The technical effects and advantages of this application are as follows:
[0080] Compared with existing technologies, this robot testing method and system based on big data and artificial intelligence adopts multimodal data acquisition and fusion technology to comprehensively capture the robot's operating status, thereby improving the comprehensiveness of the test. Compared with single-modal data, multimodal data fusion improves the test accuracy.
[0081] An innovative improved deep neural network model was proposed, including a convolutional neural network with deformable convolutional kernels and a bidirectional gated recurrent unit, which enhances the ability to extract complex features. In industrial robot weld quality inspection tasks, the recognition accuracy is improved.
[0082] The reinforcement learning-based adaptive test case generation mechanism can dynamically adjust testing strategies based on the robot's real-time performance, improving testing efficiency and coverage. In the testing of home service robots, test coverage is significantly improved compared to traditional methods.
[0083] Dynamic weight fusion algorithms can adaptively adjust weights based on the real-time performance of each model, improving the accuracy and reliability of evaluation results. In multi-model ensemble testing, dynamic weight fusion enhances evaluation results.
[0084] Introducing a test knowledge graph provides knowledge support for the testing process, enhancing the interpretability and intelligence of the system. Attached Figure Description
[0085] Figure 1 This is a schematic diagram of the method flow of this application;
[0086] Figure 2 This is a schematic diagram of the system flow of this application.
[0087] The attached figures are labeled as follows: 1. Multimodal data acquisition module; 101. Visual data acquisition unit; 102. Motion data acquisition unit; 103. Environmental data acquisition unit; 104. System status acquisition unit; 2. Data preprocessing module; 3. Deep learning evaluation module; 4. Intelligent test case generation module; 5. Result fusion module; 6. Feedback optimization module. Detailed Implementation
[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0089] Example 1
[0090] like Figure 1 The method shown is a robot testing method based on big data and artificial intelligence, which includes the following steps:
[0091] S1. Construct a multimodal test data acquisition framework to collect multimodal data of the robot in real time during the testing process;
[0092] The multimodal data in S1 includes visual data, motion data, environmental data, and system status data. The collected multimodal data is synchronized using timestamps and stored in a distributed data storage system. The data organization structure used for storing the multimodal data is as follows: ,in For multimodal datasets, For timestamps, For visual data, For motion data, For environmental data, For system status data, The total number of samples.
[0093] S2. Preprocess the collected multimodal data, including data cleaning, data standardization, and feature extraction;
[0094] S2 data cleaning uses an improved Z-score algorithm to detect outliers, and missing values are filled using an LSTM-based time series prediction model.
[0095] In data standardization, Z-score standardization is used for data that follows a Gaussian distribution, min-max standardization is used for data whose values are within a finite interval, and pixel values are normalized to the [0,1] interval for image data.
[0096] Feature extraction employs an improved autoencoder for data denoising and feature extraction. The improved autoencoder includes an encoder and a decoder, with the encoder incorporating the input data. Mapping to a low-dimensional feature space yields feature vectors. The decoder will use feature vectors Reconstructing the original data space yields reconstructed data. .
[0097] S3. Construct an improved deep neural network model to evaluate the robot's functions and performance;
[0098] The improved deep neural network models in S3 include:
[0099] An improved convolutional neural network for processing visual sensor data in robots;
[0100] An improved recurrent neural network for processing temporal motion data of robots;
[0101] The attention mechanism fusion module is used to integrate features from different modalities.
[0102] Among them, the improved convolutional neural network, namely MC-CNN, is designed for robot vision data processing. It introduces deformable convolutional kernels to capture irregular features. The specific strategy is as follows: the output feature mapping calculation formula for deformable convolution is: ,in To output feature mapping at position The value; The kernel size; For the first Each convolutional weight; For input feature mapping; , The learnable offset parameters are predicted through additional convolutional layers: ,in Predict the weights of the convolutional layer for the offset. As the activation function, the tanh function is used to limit the offset to a certain value. Within the range.
[0103] The network structure of MC-CNN is as follows:
[0104] Input layer: Receives 224×224×3 RGB images or depth images;
[0105] Deformable convolution block 1: 32 3×3 deformable convolution kernels, stride 1, padding 1, followed by BatchNorm and ReLU;
[0106] Pooling layer 1: 2×2 max pooling, step size 2;
[0107] Deformable convolution block 2: 64 3×3 deformable convolution kernels, stride 1, padding 1, followed by BatchNorm and ReLU;
[0108] Pooling layer 2: 2×2 max pooling, step size 2;
[0109] Deformable convolution block 3: 128 3×3 deformable convolution kernels, stride 1, padding 1, followed by BatchNorm and ReLU;
[0110] Global average pooling layer: converts the feature map into a 128-dimensional feature vector;
[0111] Output layer: Set the corresponding output dimensions and activation functions according to the specific task (such as object recognition, defect detection).
[0112] The improved recurrent neural network, MB-GRU, employs a bidirectional gated recurrent unit (GRU) to capture the temporal dependencies between the past and future, specifically targeting the temporal characteristics of robot motion data.
[0113] MB-GRU consists of a forward GRU and a backward GRU. The forward GRU processes data from the beginning to the end of the sequence, while the backward GRU processes data from the end to the beginning of the sequence. Finally, the hidden states from both directions are concatenated as the output.
[0114] The update formula for a single GRU unit is:
[0115] Whether to update the hidden state of the door control is as follows: ;
[0116] Reset whether the door control ignores historical hidden states, specifically: ;
[0117] The candidate hidden state is as follows: ;
[0118] The final hidden state is as follows: ;
[0119] in, To update the door; To reset the door; Currently hidden; It is the sigmoid activation function; This is element-wise multiplication; , , Enter them separately The weight matrix that acts on the update gate, reset gate, and candidate hidden states; , These are the historical hidden states. The weight matrix applied to the update gate and the reset gate; Historical state after resetting the gate filter Acts on candidate hidden state The weight matrix; , , These are the bias vectors for the update gate, reset gate, and candidate hidden state, respectively.
[0120] The output of the bidirectional GRU is:
[0121] Forward GRU hidden state ;
[0122] Backward GRU hidden state ;
[0123] Concatenate forward and backward hidden states: ;
[0124] in This is the previous forward hidden state; This is the backward hidden state at the next moment.
[0125] The network structure of MB-GRU is as follows:
[0126] Input layer: Receives a sequence of temporal feature vectors of dimension d.
[0127] Bidirectional GRU layer 1: With 64 hidden units, it returns the complete sequence.
[0128] Bidirectional GRU layer 2: with 32 hidden units, returns the complete sequence.
[0129] Temporal attention layer: Assigns attention weights to different time steps in the sequence.
[0130] Fully connected layer: Maps the output to the corresponding dimension for evaluation of robot motion stability, trajectory accuracy, etc.
[0131] The attention mechanism fusion module, or AMFM, integrates features from different modalities to learn the weights of features from different modalities. The specific strategy of the attention mechanism fusion module is as follows:
[0132] Suppose that after feature extraction, we get Feature vectors of each modality ,and ,in Indicates the first Feature vectors of each modality; The dimension of the feature vector. express A real vector of dimension;
[0133] The attention mechanism fusion process is as follows:
[0134] Feature mapping: Mapping modal features of different dimensions to the same dimensional space. ,in and For learnable parameters, For a unified feature dimension; For the first The output feature vector after mapping of each modal feature; For the first The original feature vectors of each modality; This is the weight matrix; For one A bias vector of dimension; express A dimensional real vector;
[0135] Attention weight calculation: Weights are calculated based on the similarity between features and context vectors, specifically as follows: ,in For the first Attention weights for each modality feature; These are learnable context vectors; The cosine similarity function;
[0136] Feature fusion: Weighted summation yields the final fused features. Specifically: ;
[0137] Feature enhancement: Enhancing fused features through residual connections and nonlinear transformations Specifically: ,in, This is a residual connectivity feature; and These are learnable parameters.
[0138] S4. Generate adaptive test cases based on reinforcement learning algorithms to optimize test coverage;
[0139] The specific strategy for generating adaptive test cases using reinforcement learning algorithms in S4 is as follows:
[0140] Modeling the reinforcement learning environment: Define the state space. This is represented as a combination of the robot's current state and the test environment state, specifically: ,in , , and These are feature extraction functions for visual, motion, environmental, and system state data, respectively. This is the current test coverage vector; , , and The proportion is Real-time visual, motion, environmental, and system state data; motion space A set of executable test operations. ,in For the first Each test action can be an operation, such as changing ambient lighting, adding distractions, or adjusting task difficulty. For the continuous action space, a Gaussian distribution is used to represent the action probability distribution.
[0141] reward function Configure settings to improve test coverage, determine the severity of issues discovered, and optimize test efficiency, specifically as follows: ,in The amount by which test coverage is increased; A collection of newly discovered problems; For the question The severity; Test duration; , , These are the weighting coefficients, and ; To perform the action The state after;
[0142] The reinforcement learning algorithm in S4 adopts an improved deep deterministic policy gradient algorithm, namely M-DDPG, which introduces priority experience replay and adaptive learning mechanisms.
[0143] S5. The output results of the improved deep neural network model are integrated using a dynamic weight fusion algorithm to obtain the final test conclusion;
[0144] The dynamic weights in the dynamic weight fusion algorithm used in S5 are adaptively adjusted based on the performance of each model.
[0145] S6. Generate robot performance optimization suggestions based on the test results.
[0146] In S6, performance optimization suggestions are based on test evaluation results and the constructed test knowledge graph, providing specific recommendations for optimizing the robot's hardware and software.
[0147] Example 2
[0148] Please see Figure 2 A robot testing system based on big data and artificial intelligence includes: a multimodal data acquisition module 1, used to collect various sensor data and operating status data of the robot;
[0149] Data preprocessing module 2 is used to clean, standardize, and extract features from the collected data;
[0150] Deep learning evaluation module 3 contains several improved neural network models for evaluating various aspects of the robot's performance;
[0151] Intelligent test case generation module 4 dynamically generates test cases based on reinforcement learning algorithms;
[0152] Result fusion module 5 uses a dynamic weight fusion algorithm to integrate the evaluation results of multiple models;
[0153] Feedback optimization module 6 generates robot performance optimization suggestions based on test results;
[0154] The multimodal data acquisition module 1 includes:
[0155] The visual data acquisition unit 101 includes a high-definition camera and a depth sensor for acquiring visual data;
[0156] The motion data acquisition unit 102 includes an accelerometer, a gyroscope, and a force sensor for acquiring motion data;
[0157] The environmental data acquisition unit 103 includes a temperature and humidity sensor, an air pressure sensor, and a sound sensor for collecting environmental data.
[0158] The system status acquisition unit 104 is used to collect the robot's CPU usage, memory usage, and battery status.
Claims
1. A robot testing method based on big data and artificial intelligence, characterized in that, The testing method includes the following steps: S1. Construct a multimodal test data acquisition framework to collect multimodal data of the robot in real time during the testing process; S2. Preprocess the collected multimodal data, including data cleaning, data standardization, and feature extraction; S3. Construct an improved deep neural network model to evaluate the robot's functions and performance; S4. Generate adaptive test cases based on reinforcement learning algorithms to optimize test coverage; The improved deep neural network model in S3 includes: An improved convolutional neural network for processing visual sensor data in robots; An improved recurrent neural network for processing temporal motion data of robots; The attention mechanism fusion module is used to integrate features from different modalities. Among them, the improved convolutional neural network, namely MC-CNN, is designed for robot vision data processing. It introduces deformable convolutional kernels to capture irregular features. The specific strategy is as follows: the output feature mapping calculation formula for deformable convolution is: ,in To output feature mapping at position The value; The kernel size; For the first Each convolutional weight; For input feature mapping; , The learnable offset parameters are predicted through additional convolutional layers: ,in Predict the weights of the convolutional layer for the offset. As the activation function, the tanh function is used to limit the offset to a certain value. Within the range; The improved recurrent neural network, namely MB-GRU, employs a bidirectional gated recurrent unit (GRU) to capture the temporal dependencies between the past and future, specifically targeting the temporal characteristics of robot motion data. MB-GRU consists of a forward GRU and a backward GRU. The forward GRU processes data from the beginning to the end of the sequence, while the backward GRU processes data from the end to the beginning of the sequence. Finally, the hidden states from both directions are concatenated as the output. The update formula for a single GRU unit is: Whether to update the hidden state of the door control is as follows: ; Reset whether the door control ignores historical hidden states, specifically: ; The candidate hidden state is as follows: ; The final hidden state is as follows: ; in, To update the door; To reset the door; Currently hidden; It is the sigmoid activation function; This is element-wise multiplication; , , Enter them separately The weight matrix that acts on the update gate, reset gate, and candidate hidden states; , These are the historical hidden states. The weight matrix applied to the update gate and the reset gate; Historical state after resetting the gate filter Acts on candidate hidden state The weight matrix; , , These are the bias vectors for the update gate, reset gate, and candidate hidden state, respectively. The output of the bidirectional GRU is: Forward GRU hidden state ; Backward GRU hidden state ; Concatenate forward and backward hidden states: ; in This is the previous forward hidden state; This is the backward hidden state at the next moment; The attention mechanism fusion module, namely AMFM, integrates features from different modalities to learn the weights of features from different modalities, and the specific strategy of the attention mechanism fusion module is as follows: Suppose that after feature extraction, we get Feature vectors of each modality ,and ,in Indicates the first Feature vectors of each modality; The dimension of the feature vector. express A real vector of dimension; The attention mechanism fusion process is as follows: Feature mapping: Mapping modal features of different dimensions to the same dimensional space. ,in and For learnable parameters, For a unified feature dimension; For the first The output feature vector after mapping of each modal feature; For the first The original feature vectors of each modality; This is the weight matrix; For one A bias vector of dimension; express A dimensional real vector; Attention weight calculation: Weights are calculated based on the similarity between features and context vectors, specifically as follows: ,in For the first Attention weights for each modality feature; These are learnable context vectors; The cosine similarity function; Feature fusion: Weighted summation yields the final fused features. Specifically: ; Feature enhancement: Enhancing fused features through residual connections and nonlinear transformations Specifically: ,in, This is a residual connectivity feature; and These are learnable parameters; S5. The output results of the improved deep neural network model are integrated using a dynamic weight fusion algorithm to obtain the final test conclusion; S6. Generate robot performance optimization suggestions based on the test results.
2. The robot testing method based on big data and artificial intelligence according to claim 1, characterized in that: The multimodal data in S1 includes visual data, motion data, environmental data, and system status data. The collected multimodal data is synchronized using timestamps and stored in a distributed data storage system. The data organization structure used for storing the multimodal data is as follows: ,in For multimodal datasets, For timestamps, For visual data, For motion data, For environmental data, For system status data, The total number of samples.
3. The robot testing method based on big data and artificial intelligence according to claim 2, characterized in that: The S2 data cleaning uses an improved Z-score algorithm to detect outliers, and missing values are filled using an LSTM-based time series prediction model. In data standardization, Z-score standardization is used for data that follows a Gaussian distribution, min-max standardization is used for data whose values are within a finite interval, and pixel values are normalized to the [0,1] interval for image data. Feature extraction employs an improved autoencoder for data denoising and feature extraction. The improved autoencoder includes an encoder and a decoder, with the encoder incorporating the input data. Mapping to a low-dimensional feature space yields feature vectors. ; The decoder will use feature vectors Reconstructing the original data space yields reconstructed data. .
4. The robot testing method based on big data and artificial intelligence according to claim 3, characterized in that: The specific strategy for generating adaptive test cases using the reinforcement learning algorithm in S4 is as follows: Modeling the reinforcement learning environment: Define the state space. This is represented as a combination of the robot's current state and the test environment state, specifically: ,in , , and These are feature extraction functions for visual, motion, environmental, and system state data, respectively. This is the current test coverage vector; , , and The proportion is Real-time visual, motion, environmental, and system state data; motion space A set of executable test operations. ,in For the first One test action; reward function Configure settings to improve test coverage, determine the severity of issues discovered, and optimize test efficiency, specifically as follows: ,in The amount by which test coverage is increased; A collection of newly discovered problems; For the question The severity; Test duration; , , These are the weighting coefficients, and ; To perform the action The state after; The reinforcement learning algorithm in S4 adopts an improved deep deterministic policy gradient algorithm, namely M-DDPG, which introduces priority experience replay and adaptive learning mechanisms.
5. The robot testing method based on big data and artificial intelligence according to claim 4, characterized in that: The dynamic weights in the dynamic weight fusion algorithm used in S5 are adaptively adjusted based on the performance of each model.
6. The robot testing method based on big data and artificial intelligence according to claim 5, characterized in that: The performance optimization suggestions in S6 are based on the test evaluation results and the constructed test knowledge graph, providing specific suggestions for the optimization of the robot's software and hardware.
7. A robot testing system based on big data and artificial intelligence, applied to the robot testing method based on big data and artificial intelligence as described in any one of claims 1-6, characterized in that: include: The multimodal data acquisition module is used to collect various sensor data and operational status data of the robot; The data preprocessing module is used to clean, standardize, and extract features from the collected data; The deep learning evaluation module contains several improved neural network models for evaluating various aspects of the robot's performance. The intelligent test case generation module dynamically generates test cases based on reinforcement learning algorithms. The results fusion module uses a dynamic weight fusion algorithm to integrate the evaluation results of multiple models; The feedback optimization module generates robot performance optimization suggestions based on the test results. The multimodal data acquisition module includes: The visual data acquisition unit includes a high-definition camera and a depth sensor for acquiring visual data; The motion data acquisition unit includes an accelerometer, a gyroscope, and a force sensor for acquiring motion data; The environmental data acquisition unit includes temperature and humidity sensors, air pressure sensors, and sound sensors for collecting environmental data; The system status acquisition unit is used to collect data on the robot's CPU usage, memory consumption, and battery status.
Citation Information
Patent Citations
Robot operation method based on visual and auditory fusion
CN114029963A
Intelligent behavior analysis system
CN119357888A