Data acquisition and analysis system of six-dimensional sensor

Abstract

The invention relates to the technical field of sensor detection, in particular to a data acquisition and analysis system of a six-dimensional sensor, which comprises a data acquisition module used for acquiring standard output data and auxiliary data of the six-dimensional sensor, and fusing the standard output data and the auxiliary data to construct a state vector; pre-processing and labeling the state vector; and the data processing module is used for receiving the state vector in a period of fixed time as the input of the neural network model and outputting a compensation decoupling result of the state vector. According to the method, the uncertainty in a real environment is simulated by generating the dynamically changing simulation data, and rich training samples are provided for the neural network model; in combination with a reinforcement learning model, the system can adaptively adjust the behavior of the mechanical arm in a virtual environment, and the operation of the mechanical arm under different temperature and external force conditions is optimized; the dynamic simulation and optimization training mechanism not only enhances the adaptive capacity of the mechanical arm in a complex environment, but also improves the robustness and stability of a control system of the mechanical arm.

Description

Technical Field

[0001] The present invention relates to the field of sensor detection technology, and in particular to a data acquisition and analysis system for a six-dimensional sensor. Background Art

[0002] With the rapid development of industrial automation, robotics and intelligent equipment, people's demand for multi-dimensional real-time monitoring and precise control is becoming increasingly urgent. The six-dimensional sensor combines three-axis acceleration and three-axis angular velocity (or other combined multi-dimensional sensors) to capture key information such as the spatial position, posture, speed, acceleration, etc. of the measured object in a real-time dynamic environment.

[0003] After searching, Chinese patent number CN202311042421.6 discloses a six-dimensional force / torque sensor decoupling calibration system based on reinforcement learning. It is a six-dimensional sensor calibration system for application scenarios, including an intelligent agent, an external environment and an experience pool, wherein: the intelligent agent includes a neural network structure of a Critic part and an Actor part, and a performance optimization training module; the above scheme reduces dependence on specialized calibration equipment and improves deployment flexibility by utilizing existing robotic arms and sensors to perform multi-dimensional loading and correction in specific operation tasks.

[0004] However, in automated production lines that run continuously throughout the day, robotic arms usually require high-intensity, high-frequency repetitive operations; during the online calibration process, the algorithm model requires a certain amount of time for data collection, updating and learning; if the production rhythm is strict, it is difficult to take out extra time to collect diverse force / torque data, resulting in insufficient training data required for reinforcement learning or too single data patterns; when the algorithm model works within a new action or new torque range, large deviations or instability may occur. In addition, due to the lack of sufficient diverse data, existing neural network models often cannot correctly predict forces and torques when faced with new working environments (such as different postures, temperatures or vibration conditions). This lack of generalization ability makes the robotic arm perform poorly in practical applications and cannot meet the needs of efficient and precise operations.

[0005] Therefore, a six-dimensional sensor data acquisition and analysis system is proposed to solve the above problems. Summary of the invention

[0006] Technical issues solved: In view of the above-mentioned shortcomings of the prior art, the present invention provides a data acquisition and analysis system of a six-dimensional sensor, which can effectively solve the problem that the algorithm model in the prior art is unable to take out extra time to collect diversified force / torque data in a continuously working robotic arm, resulting in insufficient training data required for reinforcement learning or too single data pattern.

[0007] Technical solution: To achieve the above objectives, the present invention is implemented through the following technical solutions: The present invention provides a data acquisition and analysis system for a six-dimensional sensor. The technical solution adopted by the present invention is as follows: comprising: a data acquisition module for acquiring standard output data and auxiliary data of the six-dimensional sensor, and fusing and constructing a state vector; then preprocessing and labeling the state vector; and further comprising: A data processing module is used to receive a state vector within a fixed period of time as an input of a neural network model, and output a compensation decoupling result of the state vector; and calculate a prediction error vector based on the compensation decoupling result; The anomaly detection module compares the predicted error vector with the preset threshold. If the predicted error vector is greater than the error threshold and lasts for N sampling cycles, it means that the sensor or the robotic arm itself has a hardware failure, or the neural network model has insufficient training data for the working condition; where N is the duration threshold for judging the abnormality; after manually eliminating the hardware failure, the model parameters of the neural network model are improved; the improvement method is: the error with abnormal data is marked as abnormal working condition data, and then the normal working condition data within a fixed historical time period is collected, and the abnormal working condition data and the normal working condition data are combined to construct mixed working condition data, and the mixed working condition data is used to asynchronously train the neural network model to obtain the optimal model parameters; The data simulation module generates a number of random posture data based on the physical model of the robotic arm; applies dynamic interference data and load data to each posture data, and fuses them into simulation data; and uses the simulation data to train the neural network model with unknown data under the optimal model parameters of the neural network model.

[0008] Among them, the output formula of the compensation decoupling result is: ; In the formula, Represents the neural network model under the initial strategy condition; is the predicted value of the force or torque of the six-dimensional sensor at time t; is the state vector; are the model parameters of the neural network; The calculation formula of the prediction error vector is: ; In the formula, is the actual measurement value of the six-dimensional sensor at time t; is the error between the predicted value and the actual measured value.

[0009] The asynchronous training method is as follows: Divide the mixed working condition data into S individuals; and randomly initialize the model parameters in each individual; Define the loss function of the neural network model: ; In the formula, is the loss function value; is the actual measurement value of the i-th individual; Represents the neural network model for the i-th individual The predicted output; Divide all individuals in the mixed working condition data into several batches; completely traverse all individuals in each batch, perform forward propagation on the model parameters of each individual, and calculate the average loss of the individual. The calculation formula is: ;Wherein, B represents the index set of the currently selected batch; is the average loss of the individuals; back-propagation is used to calculate the gradient g, and the optimization algorithm is used to update the model parameters; the training is repeated for different batches until the preset number of iterations is reached to obtain new model parameters; the evolutionary algorithm is operated on the individuals in the trained batches to obtain the next generation population; the above steps are repeated starting from the gradient update; until the loss drops to the preset threshold, or the number of iterations exceeds the set upper limit, the asynchronous training is stopped; and the optimal model parameters of the optimal individual with the highest fitness value in the next generation population are output.

[0010] The evolutionary algorithm operates in the following manner: Select the updated individual from the batch and calculate the loss function value, and use the value as the fitness of the individual; randomly select several individuals from all individuals in the mixed working condition data as candidates, compare the fitness of each individual, and mark the individual with the smallest fitness as the promoted individual; Repeat the operation several times until a sufficient number of individuals are selected as parent individuals; combine all parent individuals in pairs or multiple pairs, perform gene crossover operation to obtain the final offspring individuals; Repeat the above operation for multiple pairs of parent individuals to obtain several new batches of final offspring individuals; merge the new batch of final offspring individuals and parent individuals into a new population, and sort them in descending order according to the fitness value, and select the first few individuals from the new population as the reserved population according to the pre-set retention ratio; then randomly select from the remaining individuals to fill the remaining positions of the reserved population until the reserved population is filled, thereby forming the next generation population; Repeat the operation for the next generation of population.

[0011] In the selection process of the promoted individuals, based on the simulated annealing algorithm, one disadvantaged individual is retained in each selection process of the promoted individuals, and the retention formula is: ; In the formula, is the selection probability of the disadvantaged individual; is the fitness difference between the inferior individual and the promoted individual; T is the simulated annealing coefficient; is an exponential function.

[0012] Wherein, the gene crossover operation is performed in the following manner: Finally, temporary offspring individuals are generated by segmented crossover. After the segmented cutting is completed, the attention score of the independent segment in each temporary offspring individual is calculated based on the cumulative value of the absolute value of the gradient of each individual. The calculation formula is: ; In the formula, is the attention score of the i-th fragment F; M is the number of iterations; is the gradient value calculated at the tth iteration; the attention score of each segment is compared with the preset attention threshold, and the segment with an attention score greater than the attention threshold is selected as the key segment ; Define a small autoencoder that converts the key fragments in the parent individual and Input them into the small autoencoder to get the latent vector and the latent vector ; In the latent vector space and Perform cross operation to form a potential fusion vector ;in, is a fixed constant; the potential fusion vector is decoded and restored to the key segment space to obtain the decrypted segment ; Reuse the decrypted fragment Replace the corresponding key fragments in the temporary offspring individuals to form the final offspring individuals.

[0013] The dynamic interference data includes dynamic temperature change and dynamic external force change, and the dynamic interference data is applied in the following manner: Based on the random fluctuation function simulation, dynamic temperature changes are generated, and the generation formula is: ; In the formula, represents the temperature value at time t; is the average temperature; A represents the amplitude of temperature change; w is the frequency of temperature change; Indicates phase; Generate dynamic external force changes based on a random number generator, and the generation formula is: ; In the formula, represents the external force value at time t; is the maximum external force value; Represents a generated random number; Indicates the direction of external force.

[0014] The data simulation module constructs a dynamic change simulation model based on the reinforcement learning model, and optimizes the generation of simulation data. The optimization training method is: Define the state vector and action vector, and set the reward mechanism; The deep Q network model is defined as the basic structure of the dynamic change simulation model, which includes an input layer, a hidden layer and an output layer; the input layer is used to receive the state vector as input, and the output layer is used to receive the output of the hidden layer to calculate the Q value of each action vector; Set up a virtual scene to try different actions; the virtual scene returns a new state vector and reward based on the action vector of the robot arm; initialize the parameters of the deep Q network model and create an experience replay pool to store the action attempt records of the robot arm; Get the current state vector of the robot arm; select the action vector with random probability P and execute it to obtain the new state vector and reward; store the current state vector, action vector, new state vector and reward into the experience replay pool; randomly sample data from the experience replay pool, calculate the update formula of Q value to train the deep Q network model; stop training until the set number of training steps is reached or the adjustment effect meets the requirements, and the optimization training of the dynamic change simulation model is completed.

[0015] The optimization formula for selecting the action vector with random probability P is: ; In the formula, is the selection probability of action vector a; represents the Q value of selecting action vector a under state vector s; c is the control constant for adjusting the degree of exploration; R is the total number of selections, is the number of times the action vector a is selected; is the adjustment coefficient; For possible new action vectors The number of times selected.

[0016] The joint optimization of the adjustment coefficient and the control constant is: Set the initial adjustment coefficient and initial control constant based on historical data and record them as initial strategy parameters ; Use the initial strategy parameters to generate multiple candidate strategy parameters; The virtual scene uses each candidate strategy parameter for learning. At each time step, the effectiveness of the current candidate strategy parameter is evaluated according to the reward of each candidate strategy parameter, and the candidate strategy parameters with effectiveness are marked as valid strategy parameters; Then use the valid strategy parameters to update the initial strategy parameters to obtain the evolution strategy parameters; The update formula is: ; In the formula, is the evolution strategy parameter; is the standard deviation of the noise; H is the total number of candidate strategy parameters; is the reward of the i-th candidate strategy parameter, is the average reward of the i-th candidate strategy parameter; is the random noise of the i-th candidate strategy parameter; Repeat the above steps until the preset learning goal or convergence condition is reached, and the joint optimization of the adjustment coefficient and the control constant is completed.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. In the present invention, by collecting the standard output data of the robot arm during the working process and combining it with auxiliary data fusion to construct a state vector, a high-quality data foundation is provided for the subsequent neural network model training; and by combining the auxiliary data, it is ensured that the system can quickly and accurately obtain diversified data in the continuously working robot arm, avoiding the problems of insufficient six-dimensional sensor data collection or single pattern analysis results caused by strict production rhythm in traditional methods.

[0018] 2. In the present invention, the prediction error of the neural network model is monitored in real time through the anomaly detection module, and when an anomaly is detected, the model is asynchronously trained using mixed working condition data; this training method combines the evolutionary algorithm and the simulated annealing strategy, so that the model can be continuously iterated and optimized under complex working conditions, significantly improving the generalization ability of the model; even in a completely new posture, temperature or vibration environment, the model can still accurately predict the force or torque to ensure efficient and precise operation of the robotic arm.

[0019] 3. In the present invention, the data simulation module simulates the uncertainty in the real environment by generating dynamically changing simulation data, thereby providing rich training samples for the neural network model; combined with the reinforcement learning model, the system can adaptively adjust the behavior of the robotic arm in the virtual environment and optimize its operation under different temperature and external force conditions; this dynamic simulation and optimization training mechanism not only enhances the adaptability of the robotic arm in complex environments, but also improves the robustness and stability of its control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The figure is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] Example: Refer to Figure 1 ,This case proposes a data acquisition and analysis system of a six-dimensional sensor, including a data acquisition module, a data processing module, an omaly detection module and a data simulation module.

[0023] Among them, the data acquisition module is used to collect the standard output data of the six-dimensional sensor of the robotic arm in a preset experimental environment. The standard output data includes force or torque information in the six degrees of freedom, namely, linear force components along three orthogonal coordinate axes and torque components around three orthogonal coordinate axes.

[0024] The data processing module is used to receive standard output data as the training input of the neural network model, and perform basic calibration in a set experimental environment (temperature and humidity are relatively constant; there is no unnecessary external force interference around the robot arm area; the vibration source is relatively few or controllable) and under known loads, output the decoupling result of the standard output data (that is, the estimated value after force or torque correction), and collect a series of samples (input standard output data and output decoupling result) to form an initial data set; and input the initial data set into the neural network model, perform several rounds of iterative training, and output model parameters that meet the requirements of the default working conditions (that is, the initial weights and biases of the neural network model), which are recorded as the initial strategy.

[0025] The data acquisition module is also used to collect auxiliary data of the six-dimensional sensor itself and the key components of the robot (joints, motors or reducers, etc.), including temperature changes, vibrations, and joint angles of the robot; these data are obtained by installing auxiliary sensors around the six-dimensional sensor and the key components of the robot. When the robot is running for a long time, if the temperature of the key components is too high, it may affect the measurement accuracy of the force or torque sensor; by measuring the temperature of the key components of the robot, the thermal environment of the whole machine can be evaluated, and the compensation of data thermal drift can be achieved in combination with the standard output of the six-dimensional sensor. The vibration of the six-dimensional sensor can directly reflect the impact, jitter, or loose installation of the sensor body, especially when the robot moves at high speed or encounters a collision; once an abnormal vibration reading occurs, it may mean that the six-dimensional sensor is loosely installed, the force or torque of the robot is unstable, etc., and timely adjustment or maintenance is required; and the vibration of the key components of the robot can be used to monitor the smoothness of the robot's own movement, whether the joint reducer is worn, whether the harmonic reducer produces backlash, etc.; if abnormally high vibration is detected, it will not only affect the force or torque decoupling accuracy, but also indicate that the health of the mechanical body is worrying. The force or torque output is usually closely related to the current posture of the robot arm, load distribution, etc. When performing decoupling calibration or online reinforcement learning, it is necessary to know the current joint angle and end posture to determine the relationship between force or torque under different joint configurations.

[0026] Then, according to a unified timestamp or synchronous trigger event (for example, the robot controller triggers a six-dimensional sensor data reading every time it sends the joint angle in a cycle, and aligns the standard output data and auxiliary data at this time point), the collected standard output data and auxiliary data are merged into a state vector, and then the state vector is normalized and labeled (the labeled information is the current load and current action mode) to facilitate subsequent machine model learning; and as the robot continues to operate, the state vector will constitute a series of time series sample data.

[0027] The data processing module then receives the state vector (i.e., a series of time series sample data) within a fixed period of time as the input of the neural network model, and outputs the compensation decoupling result of the state vector; its output formula is: ; In the formula, Represents the neural network model under the initial strategy condition; is the predicted value of the force or torque of the six-dimensional sensor by the neural network model at time t, which is an output in the form of a vector, including the force or torque information in the six degrees of freedom directions; is the input of the neural network model, i.e., the state vector; is the model parameter of the neural network; the prediction error vector of the neural network model is defined, that is, the difference between the output result of the neural network model and the actual output result of the six-dimensional sensor. The calculation formula of the prediction error vector is: ; In the formula, is the actual measurement value of the six-dimensional sensor at time t; is the error between the predicted value and the actual measured value, which is also an output in the form of a vector.

[0028] The anomaly detection module is used to compare the error with the preset error threshold. For comparison, if If the sensor or the robot arm itself fails or the data is offset (for example, the sensor hardware is damaged, loose, or the wiring is not firm, or the zero point is offset due to aging; the robot arm parts are worn, the joints are loose, or a connecting rod is deformed, etc., which causes a significant change in the actual force or torque situation), or the neural network model does not have enough training data for this working condition, so that the neural network model cannot be generalized to complex scenarios (for example, a completely new posture, temperature or vibration environment, which makes the model unable to correctly predict force or torque); Indicates that the vector contained in the error is operated on by a norm, which can be a binary norm or a maximum absolute value, etc.; N is the duration threshold for judging the abnormality. When the error continues to be too large for N sampling cycles, manual inspection and elimination of hardware failures and drift are performed. If the hardware is determined to be fault-free after manual inspection, the model parameters of the neural network model are improved; the improvement method is: The errors with abnormal data are marked as abnormal working condition data, and then the normal working condition data within a fixed historical time period is collected, and the abnormal working condition data and the normal working condition data are combined to construct mixed working condition data, and the mixed working condition data is used to perform asynchronous training on the neural network model; the asynchronous training methods include: Divide the mixed working condition data into S individuals; and randomly initialize the model parameters in each individual; Define the loss function of the neural network model: ; In the formula, is the loss function value, that is, the objective function to be optimized in asynchronous training; S is the total number of individuals; is the actual measurement value of the i-th individual; Represents the neural network model for the i-th individual The predicted output; divide all individuals in the mixed working condition data into several batches (for example, each batch includes 32 individuals or 64 individuals); completely traverse all individuals in each batch, perform forward propagation on the model parameters of each individual, and calculate the average loss of the individual. The calculation formula is: ; Where B represents the index set of the currently selected batch. For example, if the batch size is 32, then B contains the indexes of 32 individuals; is the average loss of individuals; Then back-propagate to calculate the gradient g, and use optimization algorithms (such as gradient descent, Adam, etc.) to update the model parameters; repeat the training for different batches until the preset number of iterations is reached to obtain the new model parameters.

[0029] The evolutionary algorithm is operated on the batches that have completed training, and the updated individuals are selected from the batches and the loss function value is calculated, and the value is used as the fitness of the individual (the smaller the value, the better the performance). Several individuals are randomly selected from all individuals in the mixed working condition data as candidates, and the fitness of each individual is compared. The individual with the smallest fitness is marked as the promoted individual; in addition, in the selection process of promoted individuals, based on the simulated annealing algorithm, a disadvantaged individual (that is, an individual with a larger fitness value) is retained in each selection process of promoted individuals. The retention formula is: ; In the formula, is the selection probability of the disadvantaged individual; is the fitness difference between the disadvantaged individuals and the promoted individuals, The larger the value of , the more difficult it is for the inferior individuals to be accepted; T is the simulated annealing coefficient, which is used to control the degree of leniency in accepting inferior individuals. In the early stage of screening, a higher T is usually set, so that even if the fitness difference is large, the corresponding inferior individuals have a certain probability of being accepted. As the iteration proceeds, T decreases continuously, and the probability of accepting inferior individuals decreases accordingly, so that the screening process of the promoted individuals gradually focuses on the superior individuals for refinement; is an exponential function. In the selection process of the tournament, the simulated annealing algorithm is combined to increase the possibility of a small number of choices for disadvantaged individuals, so that more seemingly bad directions can be tried in the early stage, thereby further improving the chance of jumping out of the local optimum and achieving a more flexible and robust optimization process.

[0030] Repeat the operation several times until a sufficient number of individuals are selected as parents; combine all parents in pairs or multiple pairs, and perform gene crossover operations (i.e., treat the model parameters of each individual as a vector arranged in index order, and determine the position index according to a certain segmentation method, such as single-point, double-point or multi-point crossover, such as cutting at indexes p and q, dividing the vector into multiple fragments and cutting them, and then rejoining them), and finally generate temporary offspring individuals through segment crossover. After the segment cutting is completed, the attention score of the independent segment in each temporary offspring individual is calculated based on the cumulative value of the absolute value of the gradient of each individual. The calculation formula is: ; In the formula, is the attention score of the i-th fragment F; M is the size of the sliding window, that is, the number of iterations; is the gradient value calculated at the tth iteration; after comparing the attention score of each segment with the preset attention threshold, the segment with an attention score greater than the attention threshold is selected as the key segment ; Define a small autoencoder, including: , ; E is the encoder, which is used to map the input high-dimensional data (i.e., model parameters) to a more compact latent vector z, i.e., to compress the original information; D is the decoder, which is used to restore the latent vector to the same space as the model parameters and reconstruct the result as close to the original model parameters as possible; is the reconstructed model parameter after decoding; the key fragments in the parent individual and Input them into the small autoencoder respectively, and get and ; Then in the latent vector space and Perform cross operation to form a potential fusion vector ;in, is a fixed constant; finally, the potential fusion vector is decoded and restored to the key segment space to obtain the decrypted segment ; Reuse the decrypted fragment Replace the corresponding key fragments in the temporary offspring individuals to form the final offspring individuals.

[0031] A small amount of random disturbance is added to the final offspring individuals, that is, a mutation intensity parameter is set for each final offspring individual, and random noise is sampled from a normal distribution; the random noise is added to different segments of the final offspring individuals to generate new candidate solutions, so that the final offspring individuals will not only stay at the local position obtained by mixing the parent individuals, but also have the opportunity to expand to a larger range of solution space; and in the process of adding noise, an attention coefficient with a value less than 1 is set for the decrypted fragment, so that the random variation amplitude of the decrypted fragment will be smaller, thereby retaining the high-quality features formed after the fusion of the small autoencoder as much as possible, forming an evolutionary model that takes into account both robustness and innovation.

[0032] Repeat the above operation for multiple pairs of parent individuals to obtain several new batches of final offspring individuals; merge the new batch of final offspring individuals and parent individuals into a new population, and sort them in descending order according to the fitness value, and select the first few individuals from the new population as the reserved population according to the pre-set retention ratio; then randomly select from the remaining individuals to fill the remaining positions of the reserved population until the reserved population is filled, thereby forming the next generation population, including multiple optimal individuals.

[0033] Enter the next cycle, start from the gradient update, and repeat the above steps; until the loss drops to the preset threshold, or the iteration round exceeds the set upper limit, stop asynchronous training, and output the optimal result (that is, the optimal individual with the highest fitness value in the most generation population), and obtain the optimal optimal model parameters. Through asynchronous training of the neural network model, the collected abnormal data can be processed centrally, allowing the model to repeatedly iterate and learn more complex or extreme working conditions, thereby significantly improving the ability to identify and adapt to abnormal conditions, and improving the final accuracy and generalization of the model; and once new abnormal or new scene data is added in production, it can be strengthened in the next training, so that the neural network model can be continuously iterated and upgraded, and the intelligence and safety level of the production line can be continuously improved.

[0034] The data simulation module generates a certain amount of random posture data based on the existing physical model of the robotic arm (the random number generator generates several groups of joint angles in each joint); at the same time, dynamic interference data and load data are applied to each posture data; the generated data are screened to remove unreasonable or extreme postures (such as joint angles exceeding physical limits), and finally integrated into simulation data; under the optimal model parameters of the neural network model, the simulation data is used to train the neural network model with unknown data. Generating simulation data in a dynamic way can effectively simulate the uncertainty in the real environment in a virtual environment. This method can not only quickly explore a wide range of state spaces, but also provide rich training samples for the neural network model and enhance its adaptability to complex working conditions. Through the reasonable setting of dynamic temperature and external force, the generated data will be closer to the actual application scenario, providing a solid data foundation for subsequent online calibration and reinforcement learning.

[0035] The dynamic interference data includes dynamic temperature change and dynamic external force change; the dynamic temperature change is generated based on the simulation of the random fluctuation function, and the generation formula is: ; In the formula, represents the temperature value at time t; is the average temperature; A is the amplitude of temperature change, that is, the maximum range of temperature fluctuation above and below the average value; w is the frequency of temperature change; Indicates the phase, determines the starting point of temperature fluctuation, and affects the time delay of temperature fluctuation; the dynamic external force change is based on the magnitude and direction of the external force generated by the random number generator, and the generation formula is: ; In the formula, represents the external force value at time t; is the maximum external force value; Represents the generated random number, ranging from [0,1], which is used to simulate the randomness of dynamic external forces, so that the external forces have different values ​​at each time point, thereby increasing the uncertainty of the simulation; Represents the direction of the external force, usually a unit vector, indicating the direction in which the external force is applied.

[0036] During the generation of simulation data, random changes in temperature and external forces may cause the robot's control system to face uncertainty and lack of robustness. This uncertainty may lead to the following situations: when the external force or temperature changes exceed the expected range, the robot may not respond correctly, resulting in control failure or performance degradation; or the dynamically changing environment may introduce noise, affecting the accuracy of the sensor, resulting in incorrect state estimation. The data simulation module builds a dynamically changing simulation model based on the reinforcement learning model, and optimizes the generation of simulation data so that the generation of simulation data can adaptively adjust the behavior (i.e., the applied dynamic interference data and load data) in a dynamic environment. Through interaction with the environment, the robot will learn how to optimize operations under different temperature and external force conditions. The optimization training method is: Define the state vector, including the current temperature, external force, and joint angles of the robot arm; Define the action vectors, including the joint angle changes and applied loads of the robot arm; By defining the state vector and action vector, we can clearly understand the state and action mode of the robot arm that can be observed at each time step, providing an effective data basis for subsequent learning.

[0037] A reward mechanism is set up to give positive rewards when the robot arm successfully completes a task (such as grasping an object) under the current simulation data; negative rewards are given when the robot arm control fails or collides; and additional rewards are given based on the robot arm's operating efficiency (such as the shorter the time required to complete the task, the higher the reward).

[0038] The deep Q network model is defined as the basic structure of the dynamic change simulation model, which includes an input layer, a hidden layer, and an output layer. The input layer is used to receive the state vector as input. The hidden layer is used to receive the output of the input layer. The number of hidden layers is L, and each hidden layer has m nodes. The calculation formula for each node is: ; In the formula, is the output of the hidden layer, that is, the output of the jth node in the lth layer; is the activation function (e.g. ReLU); is the output of the jth node in the lth layer; Indicates the number of nodes in the l-1th layer; is the weight of the i-th node and the j-th node in the l-1th layer; is the output of the i-th node in the l-1th layer; is the bias of the jth node in the lth layer. The output layer is used to receive the output of the hidden layer and calculate the Q value of each action vector. The calculation formula is: ; In the formula, is the Q value of the jth action vector; is a linear activation function; Indicates the number of nodes in the Lth layer; is the weight of the i-th node in the L-th layer and the j-th node in the output layer; is the output of the i-th node in the L-th layer; is the bias of the jth node in the Lth layer.

[0039] Set up a virtual scene in which the robot can try different actions; the virtual scene returns a new state vector and reward based on the robot's action vector; initialize the parameters of the deep Q network model and create an experience replay pool to store the robot's action attempt records; Get the current state vector of the robot arm; select the action vector with random probability P and execute it to obtain the new state vector and reward; store the current state vector, action vector, new state vector and reward into the experience replay pool; then randomly sample data from the experience replay pool and calculate the update formula of Q value to train the deep Q network model; stop training until the set number of training steps is reached or the adjustment effect meets the requirements, thus completing the optimization training of the dynamic change simulation model.

[0040] When selecting an action vector with random probability P, the UCB strategy is combined to optimize the selection of the action vector. The optimization formula is: ; In the formula, is the selection probability of action vector a; represents the Q value of selecting action vector a under state vector s; is the exploration term, which aims to encourage the robot arm to select action vectors with fewer selections; c is the control constant for adjusting the degree of exploration, which controls the influence of the exploration term on the final selection probability; R is the total number of selections, is the number of times the action vector a is selected; is the adjustment coefficient. A high adjustment coefficient value will make the selection probability more uniform and increase the randomness of exploration; while a low adjustment coefficient value will make the selection probability more concentrated on action vectors with higher Q values, thereby reducing exploration and increasing utilization. For possible new action vectors The number of times selected.

[0041] By combining the UCB strategy and the adjustment coefficient, the robot arm can achieve a dynamic balance between exploring new action vectors and using the known best action vectors. A higher adjustment coefficient makes the selection of random probabilities more random, encouraging the exploration of different action vectors, thereby obtaining more simulation data in the simulation environment; while a lower temperature value makes the selection more inclined to action vectors with high Q values, increasing utilization and ensuring that effective decisions can be made quickly in practical applications; this balance mechanism enables the robot arm to flexibly adjust its strategy according to changes in the environment and its own learning progress to adapt to different situations.

[0042] During the optimization training of the dynamically changing simulation model, the best strategy for randomly selecting action vectors may change with time and environment, and the fixed adjustment coefficients and control constants cannot flexibly adapt to these changes, which leads to the inability to continue to improve its strategy during the optimization training process, resulting in poor performance in practical applications and failure to meet task requirements. By combining evolutionary strategies to jointly optimize the adjustment coefficients and control constants, the random selection of action vectors can converge to the optimal strategy faster and reduce unnecessary exploration; the joint optimization method is: Set the initial adjustment coefficient and initial control constant based on historical data and record them as initial strategy parameters ; Use the initial policy parameters to generate multiple candidate policy parameters by adding Gaussian noise to the initial policy parameters. The virtual scene uses each candidate policy parameter for learning. At each time step, the validity of the current candidate policy parameter is evaluated according to the reward of each candidate policy parameter, and the candidate policy parameters with validity are marked as valid policy parameters.

[0043] Then use the effective strategy parameters to update the initial strategy parameters to obtain the evolution strategy parameters; the update formula is: ; In the formula, is the evolution strategy parameter; is the standard deviation of the noise, which controls the amplitude of the noise and affects the amplitude of the initial strategy parameter update; H is the total number of candidate strategy parameters; is the reward of the i-th candidate strategy parameter, is the average reward of the i-th candidate strategy parameter; is the random noise of the i-th candidate strategy parameter.

[0044] Repeat the above steps until the preset learning goal or convergence condition is reached, and the joint optimization of the adjustment coefficient and control constant is completed. Joint optimization effectively enhances the learning ability and adaptability of the robot in complex environments. The dynamically adjusted adjustment coefficient and control constant enable the robot to achieve a better balance between exploration and utilization, thereby improving learning efficiency and decision-making quality. Through real-time updates and feedback mechanisms, the robot can flexibly respond to environmental changes and find better solutions.

[0045] The reward evaluation of candidate policy parameters is as follows: At each time step, an action training is selected for execution based on the candidate strategy parameters. After execution, a new state vector and the corresponding reward are returned based on the action vector. The rewards are accumulated over a fixed period of time, and the average reward is calculated based on the time steps over a fixed period of time and compared with the pre-set reward threshold. When the average reward is greater than the reward threshold, it means that the current candidate strategy parameters perform well and are recorded as effective strategy parameters.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data acquisition and analysis system for a six-dimensional sensor, characterized in that: include: The data acquisition module collects the standard output data and auxiliary data of the six-dimensional sensor and constructs them into a state vector; Label the state vector; also include: The data processing module receives the state vector within a fixed period of time as the input of the neural network model, and outputs the compensation decoupling result of the state vector; and calculates the prediction error vector based on the compensation decoupling result; The anomaly detection module compares the predicted error vector with a preset threshold. If the predicted error vector is greater than the error threshold and lasts for N sampling cycles, it indicates that the sensor, robotic arm or neural network model is abnormal; where N is the duration threshold for judging the abnormality; if the neural network model is abnormal, it is improved; the improvement method is: the error with abnormal data is marked as abnormal working condition data, and then the normal working condition data within a fixed historical time period is collected, and the abnormal working condition data and the normal working condition data are combined to construct mixed working condition data, and the mixed working condition data is used to asynchronously train the neural network model to obtain the optimal model parameters; The data simulation module generates a number of random posture data based on the physical model of the robotic arm; applies dynamic interference data and load data to each posture data, and fuses them into simulation data; and uses the simulation data to train the neural network model with unknown data under the optimal model parameters of the neural network model.

2. A six-dimensional sensor data acquisition and analysis system as claimed in claim 1, characterized in that: The output formula of the compensation decoupling result is: ; In the formula, Represents the neural network model under the initial policy situation; is the predicted value of the force or torque of the six-dimensional sensor at time t; is the state vector; are the model parameters of the neural network; The calculation formula of the prediction error vector is: ; In the formula, is the actual measurement value of the six-dimensional sensor at time t; is the error between the predicted value and the actual measured value.

3. A six-dimensional sensor data acquisition and analysis system as claimed in claim 2, characterized in that: The asynchronous training method is: Divide the mixed working condition data into S individuals; and randomly initialize the model parameters in each individual; Define the loss function of the neural network model: ; In the formula, is the loss function value; is the actual measurement value of the i-th individual; Represents the neural network model for the i-th individual The predicted output; Divide all individuals in the mixed working condition data into several batches; completely traverse all individuals in each batch, perform forward propagation on the model parameters of each individual, and calculate the average loss of the individual. The calculation formula is: ;Wherein, B represents the index set of the currently selected batch; is the average loss of the individuals; then back-propagate to calculate the gradient g, and use the optimization algorithm to update the model parameters; repeat the training of different batches until the preset number of iterations is reached to obtain the new model parameters; perform the evolutionary algorithm operation on the individuals in the trained batches to obtain the next generation population; repeat the above steps starting from the gradient update; until the loss drops to the preset threshold, or the number of iterations exceeds the set upper limit, stop the asynchronous training; And output the optimal model parameters of the best individual with the highest fitness value in the next generation population.

4. A six-dimensional sensor data acquisition and analysis system as claimed in claim 3, characterized in that: The evolutionary algorithm operates in the following way: Select the updated individual from the batch and calculate the loss function value, and use the value as the fitness of the individual; randomly select several individuals from all individuals in the mixed working condition data as candidates, compare the fitness of each individual, and mark the individual with the smallest fitness as the promoted individual; Repeat the operation several times until a sufficient number of individuals are selected as parent individuals; combine all parent individuals in pairs or multiple pairs, perform gene crossover operation to obtain the final offspring individuals; Repeat the above operation for multiple pairs of parent individuals to obtain several new batches of final offspring individuals; merge the new batch of final offspring individuals and parent individuals into a new population, and sort them in descending order according to the fitness value, and select the first few individuals from the new population as the reserved population according to the pre-set retention ratio; then randomly select from the remaining individuals to fill the remaining positions of the reserved population until the reserved population is filled, thereby forming the next generation population; Repeat the operation for the next generation of population.

5. A six-dimensional sensor data acquisition and analysis system as claimed in claim 4, characterized in that: In the selection process of the promoted individuals, based on the simulated annealing algorithm, one disadvantaged individual is retained in each selection process of the promoted individuals, and the retention formula is: ; In the formula, is the selection probability of the disadvantaged individual; is the fitness difference between the disadvantaged individuals and the promoted individuals; T is the simulated annealing coefficient; is an exponential function.

6. A six-dimensional sensor data acquisition and analysis system as claimed in claim 4, characterized in that: The method of the gene crossover operation is: The temporary offspring individuals are generated by segmented crossover. After the segmented cutting is completed, the attention score of the independent segment in each temporary offspring individual is calculated based on the cumulative value of the absolute value of the gradient of each individual. The calculation formula is: ; In the formula, is the attention score of the i-th fragment F; M is the number of iterations; is the gradient value calculated at the tth iteration; the attention score of each segment is compared with the preset attention threshold, and the segment with an attention score greater than the attention threshold is selected as the key segment ; Define a small autoencoder that converts the key fragments in the parent individual and Input them into the small autoencoder to get the latent vector and the latent vector ; In the latent vector space and Perform cross operation to form a potential fusion vector ;in, is a fixed constant; the potential fusion vector is decoded and restored to the key segment space to obtain the decrypted segment ; Reuse the decrypted fragment Replace the corresponding key fragments in the temporary offspring individuals to form the final offspring individuals.

7. A six-dimensional sensor data acquisition and analysis system as claimed in claim 1, characterized in that: The dynamic interference data includes dynamic temperature change and dynamic external force change, and the dynamic interference data is applied in the following manner: Based on the random fluctuation function simulation, dynamic temperature changes are generated, and the generation formula is: ; In the formula, represents the temperature value at time t; is the average temperature; A represents the amplitude of temperature change; w is the frequency of temperature change; Indicates phase; Generate dynamic external force changes based on a random number generator, and the generation formula is: ; In the formula, represents the external force value at time t; is the maximum external force value; Represents a generated random number; Indicates the direction of external force.

8. A six-dimensional sensor data acquisition and analysis system as claimed in claim 7, characterized in that: The data simulation module builds a dynamic change simulation model based on the reinforcement learning model, and optimizes the generation of simulation data. The optimization training method is: Define the state vector and action vector, and set the reward mechanism; Define the deep Q network model as the basic structure of the dynamic change simulation model, the basic structure includes an input layer, a hidden layer and an output layer; The input layer is used to receive the state vector as input, and the output layer is used to receive the output of the hidden layer to calculate the Q value of each action vector; Set up a virtual scene to try different actions; the virtual scene returns a new state vector and reward based on the action vector of the robot arm; initialize the parameters of the deep Q network model and create an experience replay pool to store the action attempt records of the robot arm; Get the current state vector of the robot arm; select an action vector with random probability P and execute it to obtain a new state vector and reward; store the current state vector, action vector, new state vector and reward in the experience replay pool; Then, data is randomly sampled from the experience replay pool, and the update formula of the Q value is calculated to train the deep Q network model; the training is stopped until the set number of training steps is reached or the adjustment effect meets the requirements, and the optimization training of the dynamically changing simulation model is completed.

9. A six-dimensional sensor data acquisition and analysis system as claimed in claim 8, characterized in that: The optimization formula for selecting the action vector with random probability P is: ; In the formula, is the selection probability of action vector a; represents the Q value of selecting action vector a under state vector s; c is the control constant for adjusting the degree of exploration; R is the total number of selections, is the number of times the action vector a is selected; is the adjustment coefficient; For possible new action vectors The number of times selected.

10. A six-dimensional sensor data acquisition and analysis system as claimed in claim 9, characterized in that: The joint optimization of the adjustment coefficient and the control constant is: Set the initial adjustment coefficient and initial control constant based on historical data and record them as initial strategy parameters ; The initial strategy parameters are used to generate multiple candidate strategy parameters. The virtual scene uses each candidate strategy parameter for learning. At each time step, the effectiveness of the current candidate strategy parameter is evaluated according to the reward of each candidate strategy parameter, and the candidate strategy parameters with effectiveness are marked as valid strategy parameters. The initial strategy parameters are then updated with the valid strategy parameters to obtain the evolution strategy parameters. The update formula is: ; In the formula, is the evolution strategy parameter; is the standard deviation of the noise; H is the total number of candidate strategy parameters; is the reward of the i-th candidate strategy parameter, is the average reward of the i-th candidate strategy parameter; is the random noise of the i-th candidate strategy parameter; Repeat the above steps until the preset learning goal or convergence condition is reached, and the joint optimization of the adjustment coefficient and the control constant is completed.

Images