A wearable device collision warning method and system based on reinforcement learning

Abstract

This invention relates to the field of reinforcement learning technology, addressing the technical problems of existing technologies that lack modeling of motion temporal patterns and individual differences, are prone to outputting false warnings or failing to warn due to insufficient generalization ability, and are unable to perceive their own prediction uncertainties. Specifically, it relates to a method and system for wearable protective gear collision warning based on reinforcement learning. The method involves: collecting first sensor data from the target protective gear; obtaining normalized data through preprocessing based on the first sensor data; seamlessly integrating enhanced state vectors that fuse temporal features and calibration probabilities for reinforcement learning, providing quantitative basis for decision-making and avoiding getting trapped in local optima; calculating importance sampling weights based on priority during sampling; and updating network parameters through gradient descent, achieving adaptive and optimizable warning detection in dynamic environments, effectively balancing the safety and false alarm rate of collision warnings.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and in particular to a method and system for collision warning of wearable protective gear based on reinforcement learning. Background Technology

[0002] Reinforcement learning allows an agent to learn the optimal strategy through trial and error in its interaction with the environment. Based on reinforcement learning, wearable protective gear collision warnings often combine multi-axis data to calculate signal vector amplitude, or use multi-stage logical judgment, such as a three-threshold method of first a large tilt, then a strong impact, and finally a stationary state, to improve accuracy. In addition, some relatively advanced traditional methods use supervised learning models, which extract time-domain and frequency-domain features from historical data to train a classifier to distinguish between collision and non-collision events.

[0003] However, existing technologies lack modeling of temporal movement patterns and individual differences, making them prone to false alarms during strenuous activities or sudden changes in posture. Furthermore, once the model is trained, the threshold tends to remain fixed, making it unable to adapt to changes in wearer behavior or new movement scenarios. In practical applications, if a wearer suddenly loses balance on a slippery surface in an unconventional scenario, the movement pattern may differ significantly from the distribution of the training data. Existing technologies may output incorrect warnings or fail to issue warnings due to insufficient generalization ability, and they cannot perceive the uncertainty of their own predictions. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a wearable device protective gear collision warning method and system based on reinforcement learning. It solves the technical problems of existing technologies, such as lack of modeling of motion time sequence patterns and individual differences, easy output of erroneous warnings or no warnings due to insufficient generalization ability, and inability to perceive the uncertainty of its own predictions.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a wearable device protective gear collision warning method based on reinforcement learning, the method comprising the following steps: collecting first sensor data of the target protective gear, and obtaining normalized data by preprocessing the first sensor data; The detection data of the target protective gear is acquired, and the normalized data and detection data are processed by Kalman gain to obtain the Kalman gain. Based on the Kalman gain and the detection data, state update and attention mechanism processing are performed to generate a multi-dimensional feature vector. Based on the multidimensional feature vectors, a probability sample set is obtained through temporal Monte Carlo sampling. The variance uncertainty of the probability sample set is then processed to obtain the enhanced state vector. Action detection is performed on the enhanced state vector based on the Q network to obtain the output action. Based on the output action, action reward processing and Q network parameter update are performed to generate the first network parameters. A motion value prediction model is constructed based on the first network parameters to obtain the predicted motion. The predicted motion is then processed in sequence with feedback and the first sensor data is updated to generate the second sensor data.

[0006] Preferably, the preprocessing based on the first sensor data includes: removing outliers from the first sensor data to obtain valid sensor data; Low-pass filtering is performed on effective sensor data to generate filtered sensor data. The filtered sensor data is normalized to obtain normalized data.

[0007] Preferably, Kalman gain processing is performed on the normalized data and the detection data, including: performing prior state prediction on the detection data to generate prior state estimates; The prior covariance is obtained by predicting the covariance based on the detection data. Kalman gain is generated by processing the prior covariance.

[0008] Preferably, the state update and attention mechanism processing based on Kalman gain and detection data includes: updating the state based on Kalman gain, prior state estimate and normalized data to obtain the posterior state estimate; Covariance is updated based on prior covariance and Kalman gain to obtain posterior covariance, and then the posterior covariance is integrated into the detection data. Multi-head self-attention processing is applied to the posterior state estimate to generate a multi-dimensional feature vector.

[0009] Preferably, the process of processing the multidimensional feature vector through temporal Monte Carlo sampling includes: performing temporal encoding processing on the multidimensional feature vector to generate the hidden state at the current time step; Based on the current hidden state, perform fully connected processing to obtain the initial collision probability; Monte Carlo sampling is performed on the initial collision probability and multidimensional feature vector to obtain a probability sample set.

[0010] Preferably, the variance uncertainty of the probability sample set is comprehensively processed, including: obtaining the average collision probability by averaging the probability sample set. Probability variance is generated by performing probability variance calculation based on the probability sample set and the average collision probability. Uncertainty calibration is performed on the average collision probability, probability variance, and multidimensional feature vector to generate an enhanced state vector.

[0011] Preferably, action detection is performed on the enhanced state vector based on the Q network, including: obtaining a multi-dimensional state vector by state encoding based on the enhanced state vector; Q-network forward propagation is performed based on multi-dimensional state vectors to generate Q-value vectors; Action recognition and detection are performed on the Q-value vector, and the output action is obtained.

[0012] Preferably, the action reward processing and Q-network parameter update based on the output action include: obtaining the action reward value and the next time step state by processing the action reward based on the output action; The multidimensional state vector, output action, action reward value and next time state are integrated into an experience tuple. Prioritize the experience tuple for experience replay to generate an optimized experience pool. The network parameters are optimized based on the optimized experience pool to generate the first network parameters.

[0013] Preferably, the predicted action is processed by feedback and the first sensor data is updated sequentially, including: generating a predicted action based on the action value prediction model, performing early warning detection processing on the predicted action, and obtaining an early warning signal; Obtain actual collision feedback, collect actual collision feedback, and generate a feedback dataset; The feedback dataset is integrated and stored, and second sensor data is generated, replacing the first sensor data with the second sensor data.

[0014] This technical solution also provides a wearable protective gear collision warning system based on reinforcement learning, which includes: The preprocessing module is used to collect the first sensor data of the target protective gear, and obtain normalized data through preprocessing based on the first sensor data; The multidimensional feature module is used to acquire the detection data of the target protective gear, perform Kalman gain processing on the normalized data and detection data to obtain the Kalman gain, and perform state update and attention mechanism processing based on the Kalman gain and detection data to generate a multidimensional feature vector. The enhanced state module is used to obtain a probability sample set by processing the multidimensional feature vector through temporal Monte Carlo sampling, and to perform variance uncertainty comprehensive processing on the probability sample set to obtain the enhanced state vector. The network parameter module is used to perform action detection on the enhanced state vector based on the Q network to obtain the output action, perform action reward processing and Q network parameter update based on the output action, and generate the first network parameters. The feedback update module is used to construct an action value prediction model based on the first network parameters, obtain the predicted action, perform feedback processing and update the first sensor data sequentially on the predicted action, and generate the second sensor data.

[0015] By employing the above technical solution, the present invention provides a method and system for collision warning of wearable protective gear based on reinforcement learning, which has at least the following beneficial effects: 1. This invention utilizes a state transition matrix and a measurement matrix to construct a system model, optimally fusing the posterior state from the previous time step with the current measurement value. This effectively suppresses sensor noise and transient anomalies, provides confidence information for reinforcement learning, and enhances decision robustness. A multi-head self-attention mechanism is introduced to extract features from the posterior state sequence of the most recent multiple time steps, providing a more discriminative state representation for the reinforcement learning agent. The combination of these two approaches ensures both the accuracy and physical consistency of state estimation and improves the depth and flexibility of temporal feature extraction, enabling collision warning to have higher sensitivity and accuracy in complex motion scenarios.

[0016] 2. This invention effectively captures the temporal dependencies of human motion through an LSTM network, outputs the current hidden state rich in contextual information, and randomly discards neurons each time through Monte Carlo dropout uncertainty estimation. Combined with uncertainty calculation, it not only obtains the center estimate of collision risk, but also perceives its uncertainty. When the variance is large, it automatically reduces the calibration probability and guides the system to take conservative actions in the ambiguous state, thereby significantly improving the robustness and safety of the early warning system in complex scenarios such as sensor noise, sudden changes in motion patterns, or missing data.

[0017] 3. This invention achieves reinforcement learning by seamlessly integrating enhanced state vectors that combine temporal features and calibration probabilities, preserving rich information and mapping states to the Q-value of each action, providing a quantitative basis for decision-making. It balances exploring new strategies with utilizing existing knowledge, avoids getting trapped in local optima, and guides the system towards a safety goal. Through a priority experience replay mechanism, importance sampling weights are calculated based on priority during sampling, and the squared loss is calculated using weighted averages. The network parameters are updated through gradient descent, achieving adaptive and optimizable early warning detection in dynamic environments, effectively balancing the safety and false alarm rate of collision warnings.

[0018] 4. This invention transforms discrete actions into physical warning signals through conditional judgment, ensuring that wearers receive immediate warnings when a collision risk occurs. By detecting collisions through collision sensors, it acquires real collision event information, forms tagged historical samples, and stores them in a database. This allows the originally unsupervised raw data to obtain supervised information from real collision markers, providing high-quality training data for subsequent offline training or online incremental learning, continuously optimizing decisions, and achieving personalized adaptive learning. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a wearable device collision warning method based on reinforcement learning according to the present invention. Figure 2 This is a structural block diagram of a wearable device protective gear collision warning system based on reinforcement learning according to the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0021] Example 1: Because existing technologies lack modeling of temporal patterns and individual differences in motion, and are prone to issuing false warnings or failing to issue warnings due to insufficient generalization ability, and are unable to perceive the uncertainty of their own predictions, please refer to... Figure 1 This embodiment provides a wearable device collision warning method based on reinforcement learning. It can avoid local optima by utilizing rich data information, achieving adaptive and optimizable warning detection, effectively improving the safety and false alarm rate of collision warnings. The method includes the following steps: S1. Collect the first sensor data of the target protective gear, and obtain normalized data through preprocessing based on the first sensor data. Existing technologies often neglect the integrity verification and outlier handling during the data reading stage, which leads to outliers caused by the sensor momentarily exceeding the range or communication packet loss being mistakenly regarded as valid information, thus triggering false alarms or missed alarms. They do not consider the changes in noise characteristics under dynamic environments, and are prone to over-smoothing and losing key collision transient characteristics, or insufficient filtering and residual noise. To solve the above problems, the specific implementation steps are as follows: S11. Based on the first sensor data, outlier removal is performed to obtain valid sensor data. The first sensor data usually contains physical quantities such as acceleration and angular velocity. In this step, the sequence number of the data packet is first checked to see if it is continuous. If a missing data packet is found, it is filled by linear interpolation based on the timestamps of the previous and next times to ensure the uniformity of the time series. For the reading of each sensor channel, a reasonable upper and lower limit threshold is set. The reading at the current time is compared with the threshold. If the reading is greater than the upper limit or less than the lower limit, it is determined to be an outlier value that exceeds the range. At this time, the valid value of the previous time is used to replace the current value. In order to further identify abrupt anomalies, the difference between the current reading and the reading at the previous time is calculated and the absolute value is taken. If the absolute value is greater than the preset maximum allowable rate of change threshold, it is also considered to be an anomaly, and the value of the previous time is used to replace it. In reinforcement learning frameworks, the stability of state inputs directly affects the effectiveness of policy learning. Therefore, statistics within a sliding window can be introduced to assist in judgment. For example, a window of length N can be selected centered on the current time, the average of all valid data within the window can be calculated, and the deviation can be obtained by subtracting the average from the current reading. At the same time, the variance of the data within the window can be calculated, and the deviation can be divided by the square root of the variance to obtain the standardized residual. If the absolute value of the residual exceeds a certain threshold, the current point is determined to be an outlier and corrected. Alternatively, the sum of squares of the data within the window can be calculated and then divided by the window length to obtain the mean square value. The square of the current reading can be compared with the mean square value, and if the deviation is too large, it can be replaced. For the case of multiple consecutive outliers, a weighted average of the valid data within the window can be used, with the weights inversely proportional to the time distance to estimate the current value, thereby avoiding error accumulation.

[0022] S12. Low-pass filtering is performed based on the effective sensor data to generate filtered sensor data. In this step, for each current moment, a filtering coefficient is first determined. This coefficient is a real number between zero and one, usually pre-selected according to the required response speed and noise level of the system. Then, the effective sensor data at the current moment is multiplied by the filtering coefficient to obtain a product term. At the same time, the filtered sensor data calculated at the previous moment is multiplied by the difference between one and the filtering coefficient to obtain another product term. Finally, the two product terms are added together, and the result is the filtered sensor data at the current moment. This process is repeated recursively. The output at each moment integrates the current observation value and the historical smooth value, so that the entire data sequence presents continuous and smooth characteristics in the time dimension, thereby effectively suppressing sudden noise interference. The final output filtered sensor data retains the main mechanical characteristics of human movement or collision process while removing unnecessary random disturbances, providing a reliable data foundation for subsequent feature extraction and collision decision.

[0023] S13. Normalize the filtered sensor data to obtain normalized data. In this step, firstly, the minimum and maximum values ​​of the sensor range need to be determined in advance. These two values ​​are usually given by the sensor hardware specifications and reflect the lower and upper limits of the physical quantities that the sensor can measure. Then, for the filtered sensor data at each moment, first perform a subtraction operation, that is, subtract the minimum value of the sensor range from the data to obtain an intermediate difference. At the same time, perform another subtraction operation, that is, subtract the minimum value from the maximum value of the sensor range to obtain another difference. This difference represents the width of the entire range. Finally, divide the intermediate difference obtained in the first step by the range width obtained in the second step. The result is the normalized sensor data at that moment.

[0024] This invention utilizes a threshold judgment process in the data reading and verification stage to perform integrity checks and outlier removal on the first sensor data. This effectively prevents abrupt data changes caused by sensor malfunctions or communication packet loss from entering subsequent processes, avoiding interference from abnormal samples on the training of the strategy network. By introducing a first-order low-pass filtering algorithm to achieve recursive smoothing, it can effectively suppress high-frequency noise and random jitter, improve the sensitivity of collision precursor patterns, eliminate the influence of different physical dimensions, accelerate the convergence speed, and enhance the generalization ability for different wearers or different sports scenarios.

[0025] S2. Acquire detection data, perform Kalman gain processing on normalized data and detection data to obtain Kalman gain, perform state update and attention mechanism processing based on Kalman gain and detection data to generate multi-dimensional feature vectors; Existing technologies often use single filtering methods or pure data-driven models, lack the ability to mine temporal deep features, are easily affected by noise and have poor interpretability. In actual violent motion scenarios, such as when the wearer suddenly falls, causing instantaneous sensor saturation or communication interruption, existing technologies may output abnormal features and trigger false warnings. To solve the above problems, the specific implementation steps are as follows: S21. Perform prior state prediction on the detection data to generate a prior state estimate. The detection data includes the previous posterior state, the previous posterior covariance, the state transition matrix, the process noise covariance, the measurement matrix, the measurement noise covariance, the current measurement value, and the input control. The initial value of the input control is 0. This step uses the previous posterior state, the previous posterior covariance, the state transition matrix, and the input control. The previous posterior state is the optimal estimate obtained through the Kalman filter update step of the previous time step. It integrates the prior prediction from the previous time step with the observation data at that time, reflecting the best estimate of the system's kinematic parameters, such as position, velocity, and acceleration, at the previous time step. The previous posterior covariance also comes from the update step of the previous time step and represents the uncertainty of the state estimate. The state transition matrix is ​​calculated by combining the Kalman gain and the observation noise covariance. It is a constant matrix pre-set according to the kinematic model of the wearable device, describing the linear relationship between the state variables and time. The control input is usually set to zero because the system has no external active control force in the collision warning scenario, so it is ignored here. In this step, the posterior state of the previous moment is first multiplied by the state transition matrix to obtain an intermediate product term. At the same time, the input control is multiplied by the control matrix. Since the control input is zero, this term is zero. Then, these two terms are added to obtain the prior state estimate of the current moment. This operation is essentially to use the system dynamic model to further predict the state. The final output prior state estimate represents the best prediction of the system state based on historical information before the arrival of the observation data at the current moment.

[0026] S22. Based on the detection data, the prior covariance is predicted using covariance. This step uses the process noise covariance in the detection data. The process noise covariance is determined by analyzing the statistical characteristics of system model errors and external random disturbances. It is usually preset as a constant diagonal matrix based on sensor noise levels, motion model accuracy, and empirical data. It can also be estimated online using an adaptive method. It represents the magnitude of uncertainty introduced by unmodeled dynamics or environmental disturbances during the state transition process. In this step, the posterior covariance of the previous time step is first multiplied on the left by the state transition matrix to obtain an intermediate matrix. Then, this intermediate matrix is ​​multiplied on the right by the transpose of the state transition matrix to obtain another intermediate matrix. Finally, this result is added to the process noise covariance to obtain the prior covariance at the current time step. In the wearable device protective gear collision warning method, the uncertainty of the previous time step is transmitted through the system dynamic model and superimposed with process noise to predict the uncertainty of the current state estimate. This ensures that the reinforcement learning agent can more accurately integrate sensor observations and model predictions based on the quantitative information of dynamic uncertainty, thereby improving the robustness and accuracy of collision warning.

[0027] S23. Perform Kalman gain processing based on prior covariance to generate Kalman gain. This step uses the measurement matrix and measurement noise covariance from the detection data. The measurement matrix is ​​a constant matrix pre-defined according to the observation model of sensors in wearable devices, such as accelerometers and gyroscopes. It describes the linear mapping relationship between system state variables, such as position, velocity, and acceleration, and the direct measurement values ​​of the sensors. It is usually determined through the sensor installation geometry and physical principles. The measurement noise covariance reflects the inherent random error characteristics of the sensor measurement process. Its value can be obtained through offline sensor calibration experiments, such as collecting a large amount of data under static conditions and calculating the variance of the measurement noise, or it can be based on the noise values ​​given in the sensor datasheet. The density parameter is estimated. In the reinforcement learning framework, this matrix is ​​usually set as a diagonal constant matrix, indicating that the noise of each measurement axis is independent. In this step, the prior covariance is first multiplied on the left by the transpose of the measurement matrix to obtain an intermediate matrix. Then, this intermediate matrix is ​​multiplied on the right by the measurement matrix to obtain another intermediate matrix. Next, this result is added to the measurement noise covariance to obtain a new matrix. Finally, the inverse operation is performed on this matrix, and the inverse matrix is ​​multiplied on the left by the product of the previously obtained prior covariance and the transpose of the measurement matrix to obtain the Kalman gain. The Kalman gain is a numerical matrix that will be used in the subsequent state update step to determine how to correct the prior state estimate based on the actual sensor observations at the current time to obtain a more accurate posterior state.

[0028] S24. Update the state based on the Kalman gain, prior state estimate, and normalized data to obtain the posterior state estimate. The normalized data used in this step is the current measurement value, which is collected in real time by sensors on the wearable device, such as accelerometers and gyroscopes, and obtained after preprocessing in S1. It reflects the direct observation of human motion or collision risk at the current moment. In this step, the measurement matrix is ​​first multiplied by the prior state estimate to obtain the measurement estimate based on model prediction. Then, the predicted value is subtracted from the current measurement value to obtain the measurement residual. The residual reflects the degree of deviation between the actual observation and the model prediction. Then, the Kalman gain is multiplied by the measurement residual to obtain a correction term. Finally, this correction term is added to the prior state estimate to obtain the posterior state estimate. The posterior state estimate is the optimal estimate of the system state at the current moment. It integrates historical information and the latest observations and has the smallest estimation error covariance for collision warning decision-making. At the same time, it will also be fed back to the Kalman filter prediction stage at the next moment to form a recursive iteration, ensuring that the protective gear can track the human body's movement state in real time and accurately warn of potential collision risks.

[0029] S25. Based on the prior covariance and Kalman gain, update the covariance to obtain the posterior covariance, and integrate the posterior covariance into the detection data. The posterior covariance can be used as the input to a new loop. This step uses the measurement matrix in the detection data. In this step, first multiply the Kalman gain by the measurement matrix to obtain an intermediate matrix. Then, construct an identity matrix with the same dimension as the prior covariance. The diagonal elements of this matrix are all one, and the remaining elements are all zero. Next, subtract the above intermediate matrix from the identity matrix to obtain another intermediate matrix. Finally, multiply this intermediate matrix on the left by the prior covariance. The prior covariance is obtained by using Kalman gain to correct the prior covariance, reflecting the reduction in uncertainty of the state estimation after incorporating the measurement information at the current moment. The posterior covariance is a quantitative value of the uncertainty of the state estimation at the current moment. It represents the confidence level of the perception of the system state after the observation update. The posterior covariance can not only be used to evaluate the reliability of the state estimation, but also help to consider the uncertainty of the current estimation in collision warning decision-making, so as to adopt a more conservative strategy when the estimation is ambiguous, thereby improving the safety and robustness of the protective gear.

[0030] S26. Perform multi-head self-attention processing on the posterior state estimate to generate a multi-dimensional feature vector. In this step, the input sequence is first multiplied by three different learnable weight matrices to obtain a query matrix, a key matrix, and a value matrix. These three matrices are used for subsequent attention calculations. Then, for each query, the dot product with all keys is calculated, and the dot product result is divided by a scaling factor, which is the square root of the key dimension, to balance gradient stability. The scaled dot product is then input into the softmax function to obtain normalized attention weights, which reflect the degree of mutual attention between time steps in the sequence. Subsequently, the attention... The weight and value matrices are weighted and summed to obtain the output of the single-head attention. This output aggregates information filtered based on relevance. To capture dependencies in different subspaces, a multi-head mechanism is used to perform the above attention calculations in parallel multiple times. Each head focuses on different feature patterns. The outputs of multiple heads are concatenated along the feature dimension to form a richer representation. This is then multiplied by a learnable fusion weight matrix to integrate the multi-head information. Finally, the fused result is passed through a fully connected layer, which first multiplies it with another learnable weight matrix and adds a bias term, and then performs a nonlinear transformation using the ReLU activation function to obtain the final multidimensional feature vector.

[0031] This invention utilizes Kalman filtering to construct a system model using state transition and measurement matrices. It optimally fuses the posterior state from the previous time step with the current measurement value, effectively suppressing sensor noise and transient anomalies. This provides confidence information for reinforcement learning, enhancing decision robustness. Furthermore, it introduces a multi-head self-attention mechanism to extract features from the posterior state sequence of recent multiple time steps, providing a more discriminative state representation for the reinforcement learning agent. The combination of these two approaches ensures both the accuracy and physical consistency of state estimation and enhances the depth and flexibility of temporal feature extraction, resulting in higher sensitivity and accuracy of collision warning in complex motion scenarios.

[0032] S3. Based on the multidimensional feature vector, a probability sample set is obtained through temporal Monte Carlo sampling. The variance uncertainty of the probability sample set is then processed to obtain the enhanced state vector. Existing technologies are prone to false alarms or missed alarms during strenuous activities or rapid changes in posture. They lack the quantification of prediction uncertainty. For example, when the wearer is engaged in high-intensity intermittent training, the sensor data may fluctuate drastically and be mixed with motion artifacts. Existing technologies may output fuzzy predictions due to feature distribution shifts. However, due to the lack of uncertainty quantification, the warning system may still blindly trigger based on this probability, leading to frequent false alarms, affecting user experience and the credibility of protective gear, and making it difficult to make adaptive detection results that conform to the current state at critical moments. To solve the above problems, the specific steps are as follows: S31. Perform temporal encoding processing based on the multidimensional feature vector to generate the current hidden state and cell state. In this step, an LSTM layer is first constructed. The multidimensional feature vector of the current time step is concatenated with the hidden state of the previous time step to form a combined vector. The initial hidden state can be set to 0 or a random feature vector can be used. The hidden state and cell state of the previous time step store historical information up to the previous time step and are the carriers for the LSTM network to maintain long-term memory. Then, the combined vector is multiplied by four different weight matrices and the corresponding bias terms are added to each to obtain four linear transformation results. Three of these results are activated by the sigmoid function to obtain the input gate, forget gate, and output gate. The control signals, whose gate values ​​are all between zero and one, are used to regulate the intensity of information flow. Another result is activated by the hyperbolic tangent function to obtain the candidate cell state, which contains the new information provided by the current input. Then, the forget gate is multiplied element-wise with the cell state of the previous time step to determine how much information to discard from historical memory. At the same time, the input gate is multiplied element-wise with the candidate cell state to determine how much new information to write into memory. The products of these two parts are added to obtain the updated cell state at the current time step. This operation realizes the iterative accumulation of long-term memory. Finally, the current cell state is compressed to between -1 and 1 by the hyperbolic tangent function and then multiplied element-wise with the output gate to obtain the hidden state at the current time step.

[0033] S32. Perform fully connected processing based on the current hidden state to obtain the preliminary collision probability. In this step, the current hidden state is first multiplied by a learnable weight matrix to obtain a linear combination result. The current hidden state condenses the temporal features and historical context information up to the current moment, which is a deep representation of human movement patterns and potential collision risks. Then, the linear combination result is added to a learnable bias term to obtain a new value. Finally, this value is input into the sigmoid activation function. The sigmoid function can compress any real number into the interval between zero and one, thereby outputting a probability value between zero and one, which is the preliminary collision probability. This probability value quantitatively reflects the likelihood of a collision at the current moment based on all historical information and current features. The closer the value is to one, the higher the collision risk; the closer it is to zero, the lower the risk.

[0034] S33. Perform Monte Carlo sampling on the initial collision probability and multidimensional feature vector to obtain a probability sample set. In this step, during the prediction phase, keep the dropout layer in the network enabled. The dropout layer is a commonly used regularization technique to prevent overfitting, which will not be elaborated here. Essentially, during each forward propagation, neurons in the network are randomly dropped with a preset probability, thereby changing the network's connection structure. The same multidimensional feature vector is repeatedly input into the LSTM encoding containing the dropout mechanism. Here, the LSTM encoding containing the dropout mechanism refers to Monte Carlo dropout sampling... In LSTM layers, dropout operations are injected, such as randomly dropping neurons in connections from input to hidden states or between layers, so that the LSTM's recurrent connections are different each time it propagates forward. In fully connected layer mapping networks, in Monte Carlo dropout sampling, this fully connected layer also randomly drops some neurons each time it propagates forward, performing a specified number of independent forward computations, such as L times. Each forward propagation is equivalent to approximating the posterior distribution of the network parameters to obtain a collision probability value. After multiple repetitions, a set of samples consisting of these probability values ​​is obtained.

[0035] This series of operations essentially uses the randomness of dropout to perform approximate Bayesian inference on the distribution predicted by the model. By sampling multiple times, statistical information about the collision probability is obtained. The final output probability sample set not only contains multiple estimates of the current collision risk, but also reflects the degree of dispersion of the model prediction. This is used to guide the agent to adopt a more conservative early warning strategy when the estimation is ambiguous, such as increasing the trigger threshold or combining other sensor data when the sample variance is large, thereby improving the robustness and safety of protective gear in complex dynamic environments.

[0036] S34. The average collision probability is obtained by averaging the probability sample set. In this step, the probabilities of all samples in the set are summed to obtain a total. Then, the total is divided by the total number of samples, which is the average collision probability. This calculation essentially integrates the results of multiple random samplings into a single comprehensive estimate, eliminating the fluctuations that may be caused by randomness in a single prediction, making the output probability smoother and more reliable. The final output average collision probability is a value between zero and one. It quantitatively represents the average level of collision risk assessed by the system at the current moment based on historical information and deep feature extraction.

[0037] S35. Based on the probability sample set and the average collision probability, perform probability variance calculation to generate the probability variance. In this step, firstly, subtract the previously calculated average collision probability from the probability of each sample to obtain the deviation of each sample. This subtraction operation reflects the degree of deviation of a single prediction from the central trend. Then, square each deviation to eliminate the influence of positive and negative signs and amplify the weight of larger deviations, resulting in a series of squared deviations. Next, sum all the squared deviations to obtain a total. Finally, divide the total by the difference between the number of samples and one, i.e., perform a division operation. The result is the probability variance. The final output probability variance is a non-negative number, which quantitatively characterizes the dispersion of the model's collision risk prediction at the current moment: the larger the variance, the more significant the differences between multiple prediction results, and the higher the uncertainty of the model regarding the current input; the smaller the variance, the more consistent the prediction results, and the more confident the model is. For example, when the variance exceeds the threshold, a conservative strategy can be adopted, such as increasing the warning threshold, to avoid false alarms or missed alarms caused by fuzzy judgment of the model, thereby enhancing the robustness and safety of protective gear in complex dynamic environments.

[0038] S36. Uncertainty calibration processing is performed on the average collision probability, probability variance, and multidimensional feature vector to generate an enhanced state vector. In this step, the square root of the probability variance is first calculated to obtain the standard deviation. Then, the standard deviation is multiplied by a preset penalty coefficient to obtain a penalty term. Next, the penalty term is subtracted from the average collision probability to obtain the calibrated collision probability. This subtraction operation reflects the risk aversion principle: when the uncertainty of the model prediction is greater, that is, the larger the variance, the calibrated probability will be reduced accordingly to avoid overly optimistic warning judgments in ambiguous states. Subsequently, the calibrated collision probability is concatenated with the multidimensional feature vector at the current time, that is, it is combined sequentially into a longer vector to obtain the enhanced state vector. This concatenation operation integrates the original temporal features with the risk estimate adjusted for uncertainty, enabling the subsequent reinforcement learning agent to simultaneously obtain motion pattern information and the confidence level of collision risk. The final output enhanced state vector is a composite representation that integrates multidimensional features and calibration probability.

[0039] This invention effectively captures the temporal dependencies of human motion through an LSTM network, outputting the current hidden state rich in contextual information. By using Monte Carlo dropout uncertainty estimation to randomly discard neurons each time, combined with uncertainty calculation, it not only obtains the center estimate of collision risk but also perceives its uncertainty. When the variance is large, it automatically reduces the calibration probability, guiding the system to take conservative actions in ambiguous states. This significantly improves the robustness and safety of the early warning system in complex scenarios such as sensor noise, sudden changes in motion patterns, or missing data.

[0040] S4. Action detection is performed on the enhanced state vector based on the Q network to obtain the output action. Based on the output action, action reward processing and Q network parameter update are performed to generate the first network parameters. Existing technologies only rely on the current instantaneous data and lack modeling of movement time sequence patterns and individual differences. They are prone to false alarms during strenuous activities or sudden changes in posture, and cannot learn and optimize from historical experience. Once the model is trained, it becomes fixed and cannot adapt to changes in wearer behavior or new movement scenarios. In practical applications, such as unconventional scenarios where wearers suddenly lose balance on slippery ground, the movement pattern may be significantly different from the distribution of training data. Existing models may output false warnings or not warn at all due to insufficient generalization ability, and cannot perceive the uncertainty of their own predictions. To solve the above problems, the specific steps are as follows: S41. Based on the enhanced state vector, a multidimensional state vector is obtained through state encoding. In this step, the input enhanced state vector is directly used as the current state without any additional mathematical transformations or operations. This means that the value of each component in the enhanced state vector is preserved intact and combined into a whole to form a multidimensional state vector. This identity mapping ensures that the rich information extracted in all previous sub-steps, including LSTM encoding, Monte Carlo dropout sampling, mean and variance calculation, uncertainty calibration, etc., can be completely and losslessly passed to subsequent sub-modules. The final output multidimensional state vector is a multidimensional real number vector that comprehensively describes the motion environment and risk level of the system at the current moment. It includes both the high-level features of the first sensor data after layers of processing and the quantitative assessment of the model's own prediction uncertainty.

[0041] S42. Based on the multidimensional state vector, perform forward propagation processing of the Q-network to generate the Q-value vector. In this step, the Q-network parameters are first constructed. The initial network parameters can be determined randomly. The parameters of each layer in the network include the weight matrix and the bias term. The multidimensional state vector is multiplied with the first layer weight matrix to obtain an intermediate result, which is then added to the bias term of that layer. Then, a nonlinear transformation is performed through the ReLU activation function to obtain the first layer hidden vector. This operation enhances the feature expression through linear combination and activation. Subsequently, the first layer hidden vector is multiplied with the second layer weight matrix, the second layer bias term is added, and the ReLU activation function is applied again to obtain the second layer hidden vector, further refining the high-level abstract features. Finally, for each optional action, such as triggering an alarm or not triggering an alarm, the corresponding output layer weight matrix is ​​multiplied with the second layer hidden vector and the respective output layer bias term is added to obtain the Q-values ​​corresponding to the two actions. These two Q-values ​​constitute a two-dimensional vector, where each component represents the estimate of the expected cumulative reward that can be obtained by performing the action in the current state.

[0042] S43. Perform action recognition and detection on the Q-value vector and obtain the output action. In this step, the current exploration rate is first input. This exploration rate is a real number between 0 and 1, and its value is dynamically determined according to the training progress. Usually, a decay strategy is adopted. In the early stage of training, a large initial value is set, such as an initial value close to 1. As the time step or training round increases, it is gradually reduced to a small constant, such as 0.01, according to the exponential decay or linear decay method. This encourages the system to fully explore the environment in the early stage, and relies more on the optimal action in the later stage. Then, a random number uniformly distributed between 0 and 1 is generated, and this random number is compared with the current exploration rate. If the random number is less than the exploration rate, an exploration action is executed, which means randomly selecting one action from all available actions as the output action. If the random number is greater than or equal to the exploration rate, an exploit action is executed, which means selecting the action corresponding to the maximum value from the Q-value vector as the output action. This process is essentially a probability-based decision branch: random selection with the exploration rate as the probability, and greedy selection with 1 minus the exploration rate as the probability. The final output action is a discrete value, for example, 0 means no warning is triggered, and 1 means a warning is triggered. This action will act on the environment, such as the protective gear performing a specific operation and obtaining a reward signal for subsequent reinforcement learning updates.

[0043] S44. Based on the output action, obtain the action reward value and the state at the next moment through action reward processing. In this step, collision feedback value is obtained through motion capture devices such as high-precision cameras or Xsens inertial capture devices. The collision feedback value is a binary value used to indicate whether a collision has actually occurred at the current moment. The specific steps are as follows: First, determine whether both the output action and the actual collision flag are positive. That is, if the action triggers a warning and a collision actually occurs, the reward value is set to +1, indicating that a correct warning receives positive incentive. If the action triggers a warning but a collision does not actually occur, the reward value is set to -1, indicating a false alarm and a slight penalty. If the action does not trigger a warning but a collision actually occurs, the reward value is set to -2, indicating a missed alarm and a heavier penalty to emphasize the harm of missed alarms. If the action does not trigger a warning and a collision actually occurs, the reward value is set to -2. If no collision occurs, the reward value is set to 0, indicating no reward or penalty for normal inaction. This reward rule directly assigns values ​​corresponding to different situations through addition and subtraction operations, guiding the reinforcement system to gradually tend towards correct warnings and avoid false alarms and missed alarms during training. At the same time, the environment returns the enhanced state vector for the next time step. This vector is calculated by processing sensor data from subsequent time steps through data preprocessing in S1, state feature extraction in S2, and collision probability prediction in S3. It integrates new motion features and calibrated collision probabilities, serving as the input for the next time step state in reinforcement learning. The final output action reward value will be used for subsequent Q-network updates. By maximizing the accumulated reward, the agent optimizes the warning strategy, and the next time step state serves as the starting point for the next round of decision-making, forming a complete interactive loop.

[0044] S45. Integrate the multidimensional state vector, output action, action reward value, and next-time state into an experience tuple. Perform priority experience replay on the experience tuple to generate an optimized experience pool. In this step, the target Q-value is first calculated using the target network: input the next-time state into the target network to obtain the Q-values ​​of all actions, take the maximum value, multiply the maximum value by the discount factor, and finally add it to the immediate reward to obtain the target Q-value. This operation integrates the current reward and the expected future return. Next, the current Q-value is calculated using the current network: input the current state and the executed actions into the current network to obtain the corresponding Q-values, and then calculate the temporal difference error, i.e., from the target Q-value to the expected future return. Subtracting the current Q value from the standard Q value yields the error value, which reflects the deviation between the current network estimate and the target. To assign priority to samples in the experience pool, the absolute value of the error is taken and a small positive constant is added to obtain the priority value. This ensures that even when the error is zero, the sample still has a basic probability of being sampled. Finally, this experience tuple and its calculated priority are stored in the experience pool to update its contents. In the priority experience replay mechanism, subsequent training will sample samples with larger temporal difference errors with greater probability based on their priority, enabling the system to learn more frequently from inaccurate predictions and accelerating policy optimization.

[0045] S46. Optimize network parameters based on the optimized experience pool to generate the first network parameters. In this step, samples are first taken from the experience pool according to priority to obtain a batch of experience tuples. Each tuple contains the current state, output action, immediate reward, and next state. For each sampled experience, its importance sampling weight is calculated. This weight is equal to the reciprocal of the total capacity of the experience pool divided by the priority of the sample. Then, it is adjusted by a power of a hyperparameter to correct the distribution bias introduced by priority sampling. Subsequently, the target Q value of each sample is calculated using the target network: the next state is input into the target network to obtain the maximum Q value of all actions. The maximum value is multiplied by the discount factor and added to the immediate reward to obtain the target value. At the same time, the current Q value is calculated using the current network: the current state and the actual action are input into the current network to obtain the corresponding estimated value. Next, the difference between the target value and the current value is calculated, and this difference is squared. The squared difference is then multiplied by the corresponding importance sampling weight to obtain the weighted squared error for each sample. The weighted squared errors of all samples are summed and divided by the batch size to obtain the weighted average loss. Finally, the gradient of this loss function with respect to the current network parameters is calculated, and this gradient is multiplied by the learning rate. The learning rate can be adaptively fine-tuned using the AdamW optimizer, a commonly used adaptive learning rate adjustment method, which will not be elaborated here. This product is subtracted from the current network parameters to update the current network parameters. Furthermore, every fixed number of training steps, the parameters of the current network are directly copied to the target value. The network, through a stable training process, ultimately outputs the first network parameters, which are the current network parameters updated by gradient descent. This allows the Q-network's estimated values ​​to more accurately reflect the true cumulative reward, thereby continuously improving the reinforcement learning agent's decision-making ability in collision warning and achieving a more precise warning triggering strategy. The current network parameters and the target network parameters are collectively referred to as the weights and biases within the neural network used to represent the Q-value function, i.e., the state-action value function, in the deep Q-network. The current network parameters refer to the network parameters that participate in decision-making and gradient updates in real time, while the target network parameters refer to the network parameters used to calculate the target Q-value, i.e., the label value in the TD target, which will not be elaborated upon here.

[0046] This invention employs reinforcement learning by seamlessly integrating enhanced state vectors that combine temporal features and calibration probabilities, preserving rich information. It utilizes a two-layer fully connected network to map states to the Q-value of each action, providing a quantitative basis for decision-making. By comparing random numbers with the probability of maximizing Q-values ​​using the exploration rate, it balances exploring new strategies with utilizing existing knowledge, avoiding getting trapped in local optima. Differentiated instant rewards are assigned to different warning results through conditional judgments, guiding the agent towards safety goals. A priority experience replay mechanism calculates importance sampling weights based on priority during sampling and weighted squared loss is calculated. Gradient descent updates network parameters, achieving adaptive and optimizable warning detection in dynamic environments, effectively balancing the safety and false alarm rate of collision warnings.

[0047] S5. Construct an action value prediction model based on the first network parameters and obtain the predicted action. Perform feedback processing and update the first sensor data on the predicted action in sequence to generate the second sensor data. Existing technology lacks the ability to trace and utilize the early warning effect, which makes it impossible for the model to be continuously improved from actual use. It also lacks real collision labels corresponding to historical data, which makes it impossible for historical data to be used for effective cyclical supervised learning. To solve the above problems, the specific implementation steps are as follows: S51. Generate predicted actions based on the action value prediction model, perform early warning detection processing on the predicted actions, and obtain early warning signals. In this step, the predicted action is a discrete value, for example, 0 means no early warning is triggered, and 1 means an early warning is triggered. The processing method is based on simple condition judgment: if the value of the predicted action is equal to 1, the early warning device on the wearable device is immediately triggered, which is usually a physical actuator such as a vibration motor, buzzer or flashlight, to send a clear warning signal to the wearer. If the value of the predicted action is equal to 0, the system remains silent and does not generate any early warning signal. This process is essentially to transform the optimal decision made by the system at each moment into a specific physical operation, which is achieved by directly comparing the action value with a preset threshold, where the preset threshold is 1.

[0048] S52. Obtain actual collision feedback and collect feedback data to generate a feedback dataset. Actual collision feedback is obtained through motion capture devices such as high-precision cameras or Xsens inertial sensors, or by directly collecting user feedback. In this step, the input actual collision feedback is first logically judged. If a collision is detected, the collision flag at the current moment is recorded as 1; otherwise, it is 0. Simultaneously, the system records the user's response to the warning signal, such as whether the user took emergency braking or evasive action after the warning. This information can be obtained through additional sensors on the device or feedback buttons on the user interface. This information is then integrated with metadata such as the current timestamp and device operating status to form a structured feedback data record. This record may contain multiple fields, such as time point, actual collision flag, user response type, and whether the warning was triggered. It is appended to the device's local storage or uploaded to a cloud database, thus constructing a continuously accumulating feedback dataset.

[0049] S53. Integrate and store the feedback dataset, and generate second sensor data, replacing the first sensor data with the second sensor data. In this step, firstly, based on the timestamp in the feedback data, locate the first sensor data corresponding to the feedback, that is, the raw readings collected by the accelerometer, gyroscope, etc. at the same time. Then, perform a merging operation: attach the feedback data as a label to the first sensor data set to form a complete labeled training sample. This operation is essentially splicing the two data sources through time alignment, so that the originally unsupervised raw data obtains supervision information with real collision markers. Subsequently, this labeled sample is stored in the historical database, that is, in the first sensor data, replacing or appending to the original raw data storage, thereby updating the historical dataset. Each first sensor data in the historical database is associated with a clear collision label, indicating whether a collision actually occurred at that time. In the subsequent offline training or online incremental learning process, when reading data, these historical data with collision labels will be read in together as supervision signals for system training or fine-tuning, providing a high-quality data foundation for continuous model optimization and personalized adaptation.

[0050] This invention transforms discrete actions into physical warning signals through conditional judgment, ensuring that wearers receive immediate alerts when a collision risk occurs. By detecting collisions through collision sensors, it acquires real collision event information, forms tagged historical samples, and stores them in a database. This allows the originally unsupervised raw data to acquire supervised information based on real collision markers, providing high-quality training data for subsequent offline training or online incremental learning, continuously optimizing decisions, and achieving personalized adaptive learning.

[0051] Example 2: Because existing technologies lack modeling of temporal patterns and individual differences, and are prone to issuing false warnings or failing to issue warnings due to insufficient generalization ability, and are unable to perceive the uncertainty of their own predictions, please refer to [link to relevant documentation]. Figure 2 The diagram shown is a structural block diagram of a wearable protective gear collision warning system based on reinforcement learning provided in this embodiment. The system includes a preprocessing module, a multi-dimensional feature module, an enhanced state module, a network parameter module, and a feedback update module. The preprocessing module is used to collect the first sensor data of the target protective gear, and obtain normalized data through preprocessing based on the first sensor data; The multidimensional feature module is used to acquire the detection data of the target protective gear, perform Kalman gain processing on the normalized data and detection data to obtain the Kalman gain, and perform state update and attention mechanism processing based on the Kalman gain and detection data to generate a multidimensional feature vector. The enhanced state module is used to obtain a probability sample set by processing the multidimensional feature vector through temporal Monte Carlo sampling, and to perform variance uncertainty comprehensive processing on the probability sample set to obtain the enhanced state vector. The network parameter module is used to perform action detection on the enhanced state vector based on the Q network to obtain the output action, perform action reward processing and Q network parameter update based on the output action, and generate the first network parameters. The feedback update module is used to construct an action value prediction model based on the first network parameters, obtain the predicted action, perform feedback processing and update the first sensor data sequentially on the predicted action, and generate the second sensor data.

[0052] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code, including but not limited to disk storage, CD-ROM, optical storage, etc.

[0053] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A wearable device collision warning method based on reinforcement learning, characterized in that, The method involves the following steps: collecting first sensor data from the target protective gear, and obtaining normalized data through preprocessing based on the first sensor data; The detection data of the target protective gear is acquired, and the normalized data and detection data are processed by Kalman gain to obtain the Kalman gain. Based on the Kalman gain and the detection data, state update and attention mechanism processing are performed to generate a multi-dimensional feature vector. Based on the multidimensional feature vectors, a probability sample set is obtained through temporal Monte Carlo sampling. The variance uncertainty of the probability sample set is then processed to obtain the enhanced state vector. Action detection is performed on the enhanced state vector based on the Q network to obtain the output action. Based on the output action, action reward processing and Q network parameter update are performed to generate the first network parameters. A motion value prediction model is constructed based on the first network parameters to obtain the predicted motion. The predicted motion is then processed in sequence with feedback and the first sensor data is updated to generate the second sensor data.

2. The wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, Preprocessing is performed based on the first sensor data, including: removing outliers from the first sensor data to obtain valid sensor data; Low-pass filtering is performed on effective sensor data to generate filtered sensor data. The filtered sensor data is normalized to obtain normalized data.

3. The wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, Kalman gain processing is applied to normalized and detection data, including: prior state prediction of detection data and generation of prior state estimates; The prior covariance is obtained by predicting the covariance based on the detection data. Kalman gain is generated by processing the prior covariance.

4. The wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, State update and attention mechanism processing based on Kalman gain and detection data include: updating the state based on Kalman gain, prior state estimate and normalized data to obtain posterior state estimate; Covariance is updated based on prior covariance and Kalman gain to obtain posterior covariance, and then the posterior covariance is integrated into the detection data. Multi-head self-attention processing is applied to the posterior state estimate to generate a multi-dimensional feature vector.

5. A wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, Based on the multidimensional feature vector, temporal Monte Carlo sampling is performed, including: temporal encoding based on the multidimensional feature vector to generate the hidden state at the current time step; Based on the current hidden state, perform fully connected processing to obtain the initial collision probability; Monte Carlo sampling is performed on the initial collision probability and multidimensional feature vector to obtain a probability sample set.

6. The wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, The variance uncertainty of the probability sample set is comprehensively processed, including: the average collision probability is obtained by averaging the probability sample set. Probability variance is generated by performing probability variance calculation based on the probability sample set and the average collision probability. Uncertainty calibration is performed on the average collision probability, probability variance, and multidimensional feature vector to generate an enhanced state vector.

7. A wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, Action detection is performed on the enhanced state vector based on the Q network, including: processing the enhanced state vector through state encoding to obtain a multi-dimensional state vector; Q-network forward propagation is performed based on multi-dimensional state vectors to generate Q-value vectors; Action recognition and detection are performed on the Q-value vector, and the output action is obtained.

8. A method for collision warning of wearable devices based on reinforcement learning according to claim 1, characterized in that, Action reward processing and Q-network parameter updates based on output actions include: obtaining action reward values ​​and the next time-step state based on the output actions through action reward processing; The multidimensional state vector, output action, action reward value and next time state are integrated into an experience tuple. Prioritize the experience tuple for experience replay to generate an optimized experience pool. The network parameters are optimized based on the optimized experience pool to generate the first network parameters.

9. A wearable device collision warning method based on reinforcement learning according to claim 1, characterized in that, The predicted action is processed sequentially with feedback and the first sensor data is updated, including: generating the predicted action based on the action value prediction model, performing early warning detection processing on the predicted action, and obtaining an early warning signal; Obtain actual collision feedback, collect actual collision feedback, and generate a feedback dataset; The feedback dataset is integrated and stored, and second sensor data is generated, replacing the first sensor data with the second sensor data.

10. A system applied to the wearable device collision warning method based on reinforcement learning as described in any one of claims 1-9, characterized in that, The system includes: The preprocessing module is used to collect the first sensor data of the target protective gear, and obtain normalized data through preprocessing based on the first sensor data; The multidimensional feature module is used to acquire the detection data of the target protective gear, perform Kalman gain processing on the normalized data and detection data to obtain the Kalman gain, and perform state update and attention mechanism processing based on the Kalman gain and detection data to generate a multidimensional feature vector. The enhanced state module is used to obtain a probability sample set by processing the multidimensional feature vector through temporal Monte Carlo sampling, and to perform variance uncertainty comprehensive processing on the probability sample set to obtain the enhanced state vector. The network parameter module is used to perform action detection on the enhanced state vector based on the Q network to obtain the output action, perform action reward processing and Q network parameter update based on the output action, and generate the first network parameters. The feedback update module is used to construct an action value prediction model based on the first network parameters, obtain the predicted action, perform feedback processing and update the first sensor data sequentially on the predicted action, and generate the second sensor data.

Images