Field operation balance car safety early warning method based on multiple sensors

Abstract

The invention discloses a field operation balance car safety early warning method based on multiple sensors, and relates to the technical field of balance car safety early warning, and the method comprises the steps: collecting multi-source sensing data and vibration signals of a field operation balance car, carrying out the preprocessing of the vibration signals, and calculating the advancing terrain gradient and confidence based on a digital elevation model; based on the preprocessed vibration signal, using a one-dimensional convolutional neural network to identify a pavement type and output a friction coefficient, combining a gradient and a confidence coefficient, obtaining a dynamic threshold through reinforcement learning and course learning strategies, and calculating a sub-risk value; and based on the sub-risk values, carrying out fusion by adopting adaptive Kalman filtering to obtain a comprehensive risk value. According to the method, the risk threshold is dynamically generated through the DRL, and the multi-source sub-risks are fused in combination with robust adaptive Kalman filtering, so that the problems that the risk threshold is rigid and the fusion result is easily interfered in a traditional method are effectively solved, and the accuracy, the adaptability and the robustness of balance car safety early warning in a field complex environment are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of safety warning technology for self-balancing scooters, and in particular to a safety warning method for self-balancing scooters used in field operations based on multiple sensors. Background Technology

[0002] As self-balancing scooters and other smart devices are increasingly used in outdoor environments, their safety warning technologies are constantly evolving—from simple alarms relying on single sensors in the early days to systems that integrate multiple pieces of information for intelligent judgment. In recent years, a common approach is to combine inertial sensors, laser ranging, GPS, and terrain elevation data to perceive the environment. Some research has also attempted to use artificial intelligence algorithms to identify road vibration signals, and then combine this with filtering or logical judgment to comprehensively assess risk. In addition, some methods use autonomous learning to dynamically adjust vehicle control strategies to adapt to complex and uneven terrain in the wild.

[0003] However, most existing technologies are designed for flat urban roads or indoor environments, and their accuracy, response speed, and stability are often significantly insufficient in the varied terrain of the field. First, the risk thresholds used are often fixed and preset, unable to be flexibly adjusted according to real-time road conditions, vehicle status, and environmental changes, easily leading to false alarms or missed alarms. Second, when integrating risk information from different sources, simple weighting or fixed rules are often used, failing to fully consider factors such as sensor noise changes over time, instantaneous equipment malfunctions, or sudden environmental interference, resulting in weak anti-interference capabilities in real, complex environments. Therefore, existing technologies generally suffer from inflexible risk assessment criteria and poor stability in fusing multi-source information. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a safety early warning method for field operation balance vehicles based on multiple sensors, which solves the problems of insufficient flexibility in risk assessment standards and poor stability of multi-source information fusion.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a safety early warning method for a field operation self-balancing vehicle based on multiple sensors, comprising,

[0008] Collect multi-source sensor data and vibration signals from the self-balancing vehicle for field operations, preprocess the vibration signals, and calculate the terrain slope and confidence level based on the digital elevation model.

[0009] Based on preprocessed vibration signals, a one-dimensional convolutional neural network is used to identify road surface types and output friction coefficients. Combining slope and confidence level, dynamic thresholds are obtained through reinforcement learning and curriculum learning strategies, and sub-risk values ​​are calculated.

[0010] Based on the sub-risk values, an adaptive Kalman filter is used to fuse them to obtain a comprehensive risk value, and hierarchical early warning and feedback learning are performed according to the comprehensive risk value.

[0011] As a preferred embodiment of the multi-sensor-based safety early warning method for self-balancing vehicles in the field described in this invention, the following steps are taken: During the operation of the self-balancing vehicle, the main controller synchronously triggers the source sensors at a fixed frequency to collect data, including the vehicle's pitch angle, roll angle, and lateral acceleration; the distance and height of obstacles ahead are collected using a time-of-flight laser rangefinder; geographical coordinates (lat, lon), travel speed, and heading angle are obtained using a high-precision GPS module and electronic compass; the horizontal accuracy factor (HDOP) is acquired simultaneously; vertical vibration signals are collected; and the vertical vibration signals are preprocessed to obtain filtered signals.

[0012] As a preferred embodiment of the multi-sensor-based field operation balance vehicle safety early warning method of the present invention, wherein: the calculation of the forward terrain slope and confidence level based on the digital elevation model refers to obtaining the geographical coordinates of the target point ahead by using a geodetic coordinate forward calculation algorithm based on the geographic coordinates (lat, lon), the heading angle and the preset forward horizontal distance. Based on the digital elevation model, the geographic coordinates (lat, lon) and the geographic coordinates of the target point in front are... Perform bilinear interpolation to obtain the elevation of the current location. Elevation of the target point The slope is calculated, and the GPS horizontal positioning error is estimated by multiplying the horizontal accuracy factor HDOP with the preset user ranging error benchmark, and the slope confidence factor is calculated.

[0013] As a preferred embodiment of the multi-sensor-based field operation balance vehicle safety early warning method of the present invention, wherein: based on the preprocessed vibration signal, a one-dimensional convolutional neural network is used to identify the road surface type and output the friction coefficient. The filtered signal is used as input data and input into a pre-trained lightweight one-dimensional convolutional neural network model to finally obtain the prediction confidence of the road surface type and the friction coefficient.

[0014] As a preferred embodiment of the multi-sensor-based field operation balance vehicle safety early warning method of the present invention, wherein: the dynamic threshold obtained by combining slope and confidence through reinforcement learning and course learning strategies refers to the state vector of the deep reinforcement learning agent composed of collected data, slope, prediction confidence, slope confidence factor and friction coefficient, which is input into the deep reinforcement learning agent, and the dynamic safety threshold is obtained through the deep reinforcement learning agent;

[0015] The deep reinforcement learning agent adopts a Double DQN architecture, comprising a master Q-network and a target Q-network. The network structure is a multilayer perceptron, and it is trained in stages in an offline simulation environment using a course learning strategy. During training, a reward enhancement mechanism is used to highlight critical security events. After each training stage, all interaction experiences are stored in an independent replay buffer, and forward propagation is performed. The master Q-network outputs the action Q-value and a dynamic threshold. The dynamic threshold is scaled and mapped using offset layers to obtain the final output value of the dynamic threshold, and a multi-objective loss function is calculated. Backpropagation and optimizer update of main Q network parameters are performed, and pruning and growth are carried out. The new sparse topology mask and corresponding weight matrix after pruning and growth are used as the main Q network parameters for the next training cycle.

[0016] Simultaneously, the target Q network synchronizes the weight matrix of the main network through soft updates and hard copies the new sparse topology mask of the main network. When the average reward in the first phase of T consecutive evaluations reaches the success rate threshold for phase switching... When the training is complete, the state vector is input into the trained DRL agent, which only performs forward inference and outputs safety control parameters, i.e., dynamic thresholds, including low-risk overturning critical angle, high-risk overturning critical angle, high-risk slope threshold, and obstacle warning distance threshold.

[0017] The corresponding weight matrix is ​​obtained through backpropagation training optimization under the updated sparse topological constraints.

[0018] As a preferred embodiment of the multi-sensor-based field operation balance vehicle safety early warning method of the present invention, the calculation of sub-risk values ​​refers to calculating sub-risk values ​​according to dynamic thresholds, including overturning risk, slippage risk, obstacle risk, and slope risk.

[0019] As a preferred embodiment of the multi-sensor-based field operation self-balancing vehicle safety early warning method described in this invention, wherein: the method of obtaining a comprehensive risk value by using adaptive Kalman filtering to fuse sub-risk values ​​refers to obtaining an initial comprehensive risk estimate and an initial error covariance during the operation of the self-balancing vehicle, and making a priori estimate of the current state based on the optimal comprehensive risk estimate and the corresponding error covariance at the previous moment;

[0020] The sub-risk values ​​at the current moment are synchronously read as four noisy observations of the true comprehensive risk, forming an observation vector. An observation matrix is ​​then constructed to calculate the innovation, and robustness processing is performed to obtain the robust innovation after all weighted suppression. Robust innovations after weighted suppression Calculate the sample covariance matrix And update the observation noise covariance;

[0021] Adaptive Kalman fusion is performed using the updated observation noise covariance to obtain an updated comprehensive risk estimate. The error covariance is used to estimate the updated overall risk. Physical limiting is applied, and a weighted root mean square estimation is performed to obtain the optimal comprehensive risk estimate. That is, the comprehensive risk value.

[0022] As a preferred embodiment of the multi-sensor-based field operation balance vehicle safety early warning method of the present invention, wherein: the step of performing hierarchical early warning and feedback learning based on the comprehensive risk value refers to... The size is set, three response levels are preset, and different early warning outputs and control interventions are adopted. After each control intervention is executed, the key information is packaged and recorded, and the key information is combined into a reinforcement learning standard experience tuple for optimization training.

[0023] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the multi-sensor-based field operation balance vehicle safety warning method described in the first aspect of the present invention.

[0024] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the safety warning method for a field operation balance vehicle based on multiple sensors as described in the first aspect of the present invention.

[0025] The beneficial effects of this invention are as follows: By constructing a deep reinforcement learning (DRL) agent, this invention takes multi-source state information (including slope, friction coefficient, obstacle features, and confidence factors) as input and directly generates physically meaningful dynamic thresholds (such as overturning critical angle, high-risk slope threshold, obstacle warning distance, etc.) using a dual-output head structure. This enables the risk judgment boundary to adaptively evolve with the environment and vehicle state. At the same time, through robust suppression and adaptive estimation of observation noise based on sample covariance, an adaptive Kalman filter framework is constructed to optimally fuse the four types of sub-risks. This effectively overcomes the problem that traditional static thresholds are prone to false alarms or missed alarms in complex field scenarios and significantly improves the robustness of multi-source risk fusion to sensor instantaneous failures and environmental changes. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the safety early warning method for a field operation self-balancing vehicle based on multiple sensors in Example 1.

[0028] Figure 2 This is a flowchart of the adaptive robust risk fusion method in Example 1. Detailed Implementation

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0032] Example 1, referring to Figure 1 and Figure 2This is the first embodiment of the present invention, which provides a safety early warning method for a field operation self-balancing vehicle based on multiple sensors, including the following steps:

[0033] S1. Collect multi-source sensor data and vibration signals of the field operation balance vehicle, preprocess the vibration signals, and calculate the forward terrain slope and confidence level based on the digital elevation model.

[0034] Specifically, the process involves collecting multi-source sensor data and vibration signals from the self-balancing scooter for field operations, and preprocessing the vibration signals. During scooter operation, the main controller synchronously triggers the source sensors at a fixed frequency to collect data, including acquiring the vehicle's pitch angle via a six-axis inertial measurement unit (IMU). Roll angle and lateral acceleration The distance to obstacles ahead is collected using a time-of-flight (ToF) laser rangefinder. With height Geographic coordinates (lat, lon), speed v, and heading angle are obtained through a high-precision GNSS (Global Navigation Satellite System, including GPS, etc.) module and electronic compass. Simultaneously, the horizontal accuracy factor HDOP is acquired to quantify the impact of the current visible satellite geometry distribution on horizontal positioning accuracy (latitude, longitude). Vertical vibration signals are collected using a vertical acceleration sensor at a frequency greater than or equal to the system's master control frequency. ;

[0035] The system's main control frequency is achieved through a microcontroller's internal timer interrupt mechanism to schedule periodic tasks, ensuring that data from each sensor is collected synchronously.

[0036] All sensor data is aligned using hardware timestamps (based on the IMU sampling time, with linear interpolation used for low-frequency signals such as GPS for synchronization), and vertical vibration signals are also synchronized. Digital bandpass filtering (using a second-order IIR Butterworth structure, implemented by bilinear transform from analog prototype to digital domain with a sampling rate of 100 Hz) is performed to reflect the frequency band components of typical road surface excitation, filtering out low-frequency drift of the motor and high-frequency electromagnetic interference to obtain the filtered signal. The data is then normalized to zero mean and unit variance before being used as input to a subsequent one-dimensional convolutional neural network (1D-CNN).

[0037] By using multi-source sensor synchronous triggering acquisition and hardware timestamp alignment technology, the high accuracy and real-time performance of environmental perception data are ensured. The microcontroller timer interrupt mechanism ensures stable system operation. Through digital bandpass filtering and standardization processing, the quality of vibration signals is significantly improved, which not only enhances the safety of the self-balancing vehicle in complex terrain, but also enhances the accuracy and robustness of risk assessment.

[0038] Furthermore, the calculation of the forward terrain slope and confidence level based on the digital elevation model is based on geographic coordinates (lat, lon) and heading angle. The geographic coordinates of the target point ahead are calculated using a geodetic coordinate forward calculation algorithm, based on a preset forward horizontal distance d (set according to the physical performance limitations of the sensor, with a typical range of [0.3m, 2.5m]).

[0039]

[0040] in, is the geographical coordinate of the target point ahead, and B is the average radius of the Earth;

[0041] Based on a locally stored regular grid digital elevation model (DEM), the geographic coordinates (lat, lon) and the geographic coordinates of the target point in front are... Perform bilinear interpolation to obtain the elevation of the current location. Elevation of the target point Based on the current location elevation Target point elevation And the slope is calculated based on the preset forward horizontal distance d:

[0042]

[0043] Where s is the slope, and arctan(·) is the arctangent function. It is pi;

[0044] The GPS horizontal positioning error is estimated by multiplying the horizontal accuracy factor HDOP by a preset user ranging error benchmark (a typical error level of GNSS receiver pseudorange measurement under specific environments (such as open sky, urban canyons, and forest areas), with a value of 0.8m, based on the typical user ranging error (URE) specified in the receiver manufacturer's technical manual or the ICD-GPS-200 standard). And calculate the slope confidence factor:

[0045]

[0046] in, is the slope confidence factor, k is the attenuation coefficient, used to adjust the confidence level's sensitivity to positioning errors. It is determined based on the statistical distribution of GNSS positioning errors and security strategy requirements, and its value range is [0.20, 0.25]. exp(·) is the natural exponential function.

[0047] By integrating real-time positioning, heading, and digital elevation models, a forward-looking estimate of the terrain slope ahead was achieved. At the same time, a slope confidence factor was constructed based on HDOP and measured error benchmarks to quantify the impact of GNSS positioning uncertainty on slope calculation. The synergistic effect of the two not only improves the initiative and accuracy of terrain perception in complex field environments, but also significantly enhances the anti-interference capability and decision reliability of the safety early warning system, effectively avoiding false alarms and missed alarms.

[0048] S2. Based on the preprocessed vibration signal, a one-dimensional convolutional neural network is used to identify the road surface type and output the friction coefficient. Combined with the slope and confidence level, a dynamic threshold is obtained through reinforcement learning and curriculum learning strategies, and the sub-risk value is calculated.

[0049] Specifically, based on the preprocessed vibration signal, a one-dimensional convolutional neural network is used to identify the road surface type and output the friction coefficient index of the filtered signal. As input data, the data is processed by a pre-trained lightweight one-dimensional convolutional neural network (1D-CNN) model to ultimately obtain the prediction confidence scores for road surface types (hard asphalt, loose sand, slippery mud, compacted gravel, ice, etc.). (A larger value indicates that the network is more certain that the road surface type is the actual situation) and friction coefficient ;

[0050] The lightweight one-dimensional convolutional neural network (1D-CNN) model includes:

[0051] Input layer: Receives the filtered signal ;

[0052] Convolutional layer 1: 32 convolutional kernels, kernel size 5, stride 1, activation function ReLU, used to extract low-level features of time series signals;

[0053] Max pooling layer: The pooling size is 2 and the stride is 2. It is used to reduce the feature dimension and the amount of computation, and increase the robustness of the model.

[0054] Convolutional layer 2: 64 convolutional kernels, kernel size 3, stride 1, activation function ReLU, used to further extract deeper features;

[0055] Global average pooling layer: Performs global average pooling on the feature map output by convolutional layer 2, reducing its dimensionality to a single vector;

[0056] Fully connected layer: Contains 5 neurons, used to output the category score for each road surface type;

[0057] Dual-output head structure: includes a classification head and a regression head. The classification head contains a fully connected layer with 5 neurons, followed by a Softmax layer, which outputs the probability distribution of the road surface, i.e. the prediction confidence. The regression head contains a fully connected layer with 1 neuron (without an activation function or with linear activation), which directly outputs the estimated value of the friction coefficient.

[0058] The model uses a multi-task joint loss function for supervised training, with Adam as the optimizer. The training process is completed offline in the simulation environment. The training labels include manually labeled road surface types and corresponding measured friction coefficients.

[0059] The multi-task joint loss function includes cross-entropy loss for road surface classification and mean squared error (MSE) loss for friction coefficient regression.

[0060] By using a lightweight 1D-CNN to perform end-side inference on the filtered vibration signal, high-precision identification and simultaneous estimation of friction coefficients for various types of outdoor road surfaces are achieved. The output prediction confidence further supports the reliability weighting of risk assessment. Without additional sensors, the vehicle's perception, adaptability, and safety control accuracy under complex ground conditions are significantly improved.

[0061] Furthermore, by combining slope and confidence level, a dynamic threshold is obtained through reinforcement learning and curriculum learning strategies, which includes collected data, slope s, and predicted confidence level. Slope confidence factor coefficient of friction The state vector that makes up a deep reinforcement learning (DRL) agent:

[0062]

[0063] Where x is the state vector and v is the travel speed. It is the vehicle pitch angle, It's the roll angle. It is lateral acceleration. , It refers to the distance and height of the obstacle ahead;

[0064] A deep reinforcement learning (DRL) agent outputs a dynamic security threshold based on the current state vector. This DRL agent employs a multi-head output (Double) mechanism. The DQN architecture (comprising a main Q-network (for action selection and loss calculation) and a target Q-network (for providing a stable Bellman objective)) uses a multilayer perceptron (MLP) structure. The network structure is determined and fixed during offline training to adapt to the memory and computing power limitations of the onboard MCU. Specifically, the network structure includes an input layer (9 neurons, receiving state vectors), a shared hidden layer (16 neurons, using the ReLU activation function), and a dual-output head structure (an action value head (output dimension equal to the preset action space size, used for policy decision-making) and a dynamic threshold head (containing 4 neurons, each directly outputting physically meaningful safety control parameters). To ensure the physical rationality of the outputs, each threshold output is constrained to a preset safety range by independent scaling and offset layers (based on general safety standards in the electric scooter industry, instability boundary data obtained from actual vehicle bench testing, and human-machine interaction safety margins; preset safety ranges for low-risk overturning angle, high-risk overturning angle, high-risk slope threshold, and obstacle warning distance threshold are set respectively). , , [0.3m, 2.5m]);

[0065] To reduce computational overhead, the Multilayer Perceptron (MLP) employs a fixed sparse topology. This topology is initialized during the initial training phase using the Static Sparse Evolutionary Training (SET) method, dynamically optimized during training, and finally fixed before deployment. Specifically, the number of connections in layer l (connecting layer l-1 to layer l) is set as follows:

[0066]

[0067] in, It is the number of connections in layer l. , These represent the number of neurons in layer (l-1) and layer l, respectively. The hyperparameters controlling the sparsity of the l-th layer are determined based on multiple rounds of simulation experiments. Candidate network structures with different hyperparameter values ​​are constructed in the simulation environment. The average return rate (policy performance) and single-step inference time of each structure are evaluated under standard test scenarios. The hyperparameter value with the best overall performance is selected as the final configuration and is fixed into the main Q network topology mask before deployment. The value range is [0.3, 0.5], which achieves a balance between model accuracy and MCU inference speed.

[0068] Randomly and uniformly select from all positions. Each connection is established to form an initial sparse mask, which is then trained. During the training process (completed offline in a simulation environment), the course is pre-divided into three stages based on the risk complexity of the actual driving environment of the self-balancing scooter:

[0069] Phase 1 Flat and dry road surface, meeting the slope requirements. And the coefficient of friction ;

[0070] Phase Two Slightly sloping, slippery road surface, meeting the slope requirements. And the coefficient of friction ;

[0071] Phase Three : steep slopes or complex obstacle roads, meeting the slope requirements or obstacle height ,in, , These are the lower and upper limits of friction on dry roads, respectively. Based on actual vehicle testing, the lower limit ranges from [0.5, 0.7], and the upper limit ranges from [0.8, 1.0]. This is the height of high-risk obstacles, set based on the wheel radius. This is a set basic high-risk slope threshold, determined through actual vehicle climbing ability testing, with a value range of [value range missing]. , This is the slope risk initiation angle, set with reference to the general safety standards in the electric self-balancing scooter industry, and its value range is [value range missing]. ;

[0072] The agent in the first stage Interact with the simulation environment in the scene, and receive an initial reward for each interaction. To highlight key learning events (such as the first successful obstacle avoidance or a sudden collision), the original reward is... Obtained by cropping and normalization And calculate the relative rate of change based on the magnitude of the change between the current reward and the reward at the previous moment:

[0073]

[0074] in, It is the relative rate of change. It is a small constant used to prevent the denominator from being zero. It is the normalized reward from the previous moment;

[0075] When relative rate of change The absolute value exceeds the preset relative change rate threshold. (By setting up a grid search, running the DRL agent in a simulation environment, recording the normalized reward for each step, calculating the relative rate of change, trying different relative rate of change thresholds, and observing the relative rate of change threshold that can effectively capture real key events (such as successful first obstacle avoidance) and avoid false triggering (avoiding frequent enhancements due to noise), the relative rate of change threshold that makes the policy converge quickly and has high sample efficiency is selected, with a value range of [0.1, 0.5]), then a sudden change in behavioral consequences occurs (such as suddenly changing from "safe passage" to "collision", or successful first crossing of a narrow passage), and the current reward is enhanced:

[0076]

[0077] in, It's an enhanced reward. It is an increase in quantity;

[0078] Increase The calculation formula is:

[0079]

[0080] in, The enhancement strength coefficient (determined in simulation through grid search, with a fixed relative rate of change threshold, different enhancement strength coefficients are tried, and multiple rounds of training are conducted to record the average reward, convergence speed, and whether reward explosion or oscillation occurs, and the enhancement strength coefficient that enables the agent to quickly learn key skills without diverging is selected, with a value range of [0.5, 2.0]), and sgn(·) is the sign function, which ensures that the direction of reward enhancement is consistent with the trend of change;

[0081] After each training phase, all interaction experiences (state, action, reward, next state) are stored in a separate replay buffer. For each sampled instance, forward propagation is performed, and the main Q-network outputs the action Q-value and dynamic threshold. To ensure the physical validity of the output, each dynamic threshold is scaled and offset layer mapped.

[0082]

[0083] in, This is the final output value of the f-th dynamic threshold, and clip(·) is the truncation function. It is the f-th dynamic threshold output by the dynamic threshold header. It is the Sigmoid function. It is a preset safety zone;

[0084] When training reaches stage y (y=1,2,3), an equal number of samples are randomly drawn from the buffer, and the multi-objective loss function is calculated:

[0085]

[0086] in, It is a multi-objective loss function. is the discount factor, set experimentally (running the DRL agent in a simulation environment, trying different discount factors, observing whether the training results converge, the convergence speed, and whether "delayed rewards cannot be learned" or "over-reliance on short-term behavior" occurs; selecting the discount factor that allows the policy to learn stably in all three stages of the course, with a value range of [0.9, 0.99]). D is the replay buffer. It is the state vector at the next moment. It is the action vector for the next moment. It is the maximum future Q value estimated by the target network. It is the Q-value estimate of the main network for the current state-action pair. These are all dynamic thresholds output by the dynamic threshold header, namely, the low-risk overturning critical angle, the high-risk overturning critical angle, the high-risk slope threshold, and the obstacle warning distance threshold. These are ideal dynamic threshold supervision labels, generated in real time by the simulation environment. The multi-task weights are determined through grid search (while fixing other hyperparameters (such as discount factors), trying different multi-task weights, training the model and evaluating the performance of the action policy (average reward), the reasonableness of the threshold output (whether it falls within the preset safe range), and whether "threshold drift" or "action rigidity" occurs; the multi-task weights that optimize both together without significant deviation are selected, with a value range of [0.1, 2.0]). E[·] represents the mathematical expectation of the batch of samples sampled from the playback buffer D. It is the square of the L2 norm;

[0087] Under the current sparse topological mask constraint, the multi-objective loss function Backpropagation and optimizer updates of the master Q network parameters are performed. Every e environment interaction steps (i.e., e transition samples are stored in the replay buffer), a topology adaptation operation is performed, including pruning (for each network layer, removing the layer with the smallest absolute weight). The existing connections of the proportion, where, The pruning ratio controls the sparsity intensity and is determined in the simulation environment through grid search. By performing a grid search on candidate pruning ratio values ​​in the simulation environment, the average reward, convergence speed, and training stability under each configuration are comprehensively evaluated, and the optimal value that balances model accuracy and MCU inference speed is selected, with a value range of [0.05, 0.25]. Growth (in the same layer, randomly selecting new connections equal to the number of pruning branches from unconnected positions (i.e., maintaining the total number of connections)) and growth (in the same layer, randomly selecting new connections equal to the number of pruning branches from unconnected positions) are also discussed. (constant), and initialize the new connection weights to 0);

[0088] The pruned and grown new sparse topology mask and corresponding weight matrix (optimized through backpropagation training under the updated sparse topology constraints) are used as the parameters of the main Q-network in the next training cycle. Simultaneously, the target Q-network synchronizes the main network's weight matrix through soft updates and hard copies the main network's new sparse topology mask to maintain topology consistency. The success rate threshold for stage switching is reached when the average reward of the DRL agent in each of the T consecutive evaluations in each stage reaches that threshold. (Through Bayesian optimization, an initial success rate threshold is selected. Each test uses the success rate threshold most likely to improve performance. A Gaussian process is used to model the performance response, updating the posterior distribution to guide the next trial direction. Iteration or performance convergence is used to find the optimal success rate threshold that allows the policy to learn stably in all three stages, and repeated testing is performed. The value range is [0.6, 0.9]). When the skill for that stage is mastered, the next stage begins. After training, the final sparse topological mask and weights are fixed. The state vector x is input into the trained DRL agent, which only performs forward inference and outputs physically meaningful safety control parameters, i.e., dynamic safety thresholds, including the low-risk overturning angle. Overturning high-risk critical angle High-risk slope threshold Obstacle warning distance threshold .

[0089] By constructing a state vector that integrates multi-source perception and confidence, a lightweight sparse DRL agent is driven to dynamically output physically meaningful safety thresholds. This solves the problem that traditional fixed thresholds cannot adapt to complex field conditions. By adopting a course learning and reward enhancement mechanism, the efficiency of policy training and generalization ability are significantly improved. Through sparse topology and MLP structure design, the model can be inferred in real time on the vehicle MCU. Multi-objective loss functions jointly optimize actions and thresholds, so that the output parameters have both decision performance and engineering interpretability. Overall, the safety warning has achieved a leap from "passive response" to "active prediction-adaptive decision-making". While ensuring the lightweight deployment of the system, it has greatly improved the risk perception accuracy, control robustness and human-machine collaborative safety of the field operation balance vehicle.

[0090] Furthermore, calculating sub-risk values ​​refers to calculating sub-risk values ​​separately based on dynamic safety thresholds, including the risk of overturning. Risk of slipping Obstacles and risks Slope risk ;

[0091] The risk of overturning The calculation formula is:

[0092]

[0093] in, It is the maximum attitude angle. , These are the low-risk and high-risk overturning critical angles, which are dynamically output through the DRL agent.

[0094] The calculation formula is:

[0095]

[0096]

[0097] in, It is the vehicle's pitch angle. It's the roll angle. It is lateral acceleration. It is the equivalent roll angle. The adjusted roll angle is obtained by adding the equivalent roll angle to the actual roll angle. It is gravitational acceleration;

[0098] The risk of slipping The calculation formula is:

[0099]

[0100] The obstacle risk The calculation formula is:

[0101]

[0102] in, It is the obstacle warning distance threshold, which is dynamically output through the DRL agent. It is the distance to the obstacle. It is the height of the obstacle. It is the minimum effective obstacle distance, determined by the specifications of the ToF sensor and the length of the vehicle. This is the obstacle risk weight, set according to FMEA (Failure Mode and Effects Analysis), with a value range of [1.0, 3.0]. The obstacle height is negligible, determined through real-vehicle testing. This is the height of high-risk obstacles, set based on the wheel radius;

[0103] The slope risk The calculation formula is:

[0104]

[0105] in, This is a high-risk slope threshold, dynamically output through a DRL agent. This is the slope risk initiation angle, a commonly used safety standard in the electric self-balancing scooter industry, with a value range of [value missing]. .

[0106] The system uses DRL dynamic thresholds to drive refined modeling of four types of sub-risks: overturning risk achieves continuous hierarchical assessment of posture, slippage risk is directly related to the friction coefficient, obstacle risk integrates distance, height and failure weight for scenario discrimination, and slope risk adaptively matches vehicle capabilities and road conditions. The calculation logic of each sub-risk has both physical interpretability and environmental adaptability, which together support a comprehensive safety early warning system with high robustness and low false alarms, significantly improving the active protection capability of the self-balancing scooter in complex outdoor environments.

[0107] S3. Based on the sub-risk values, an adaptive Kalman filter is used to fuse them to obtain a comprehensive risk value. Based on the comprehensive risk value, hierarchical early warning and feedback learning are performed.

[0108] Specifically, the comprehensive risk value obtained by fusing sub-risk values ​​using adaptive Kalman filtering refers to the initial comprehensive risk estimate obtained through multiple cold start experiments calibrated under standard test environments (flat, dry, and unobstructed surfaces) during the operation of the self-balancing scooter. and initial error covariance Based on the optimal comprehensive risk estimate of the previous moment and the corresponding error covariance (when At that time, taking the combined risk estimate and error covariance as initial values, a priori estimate is made for the current state:

[0109]

[0110]

[0111] in, It is a priori estimate of the overall risk. It is the prior estimation error covariance, and Q is the process noise variance, reflecting the uncertainty of the overall risk over time;

[0112] The sub-risk values ​​at the current moment are read synchronously as four noisy observations of the true comprehensive risk, and these observations are combined to form an observation vector:

[0113]

[0114] in, It is the observation vector;

[0115] Construct an observation matrix G, representing that each sub-risk linearly observes the same comprehensive risk, and calculate the innovation:

[0116]

[0117] in, It is new information. It is a priori estimate of the overall risk;

[0118] To suppress transient sensor failures or sudden environmental changes (such as false detections by ToF or IMU surges), information... To achieve robustness, maintain a new information history buffer of length O. ,in, It is the first The innovation vector at time step is the difference between the observed and predicted values. c is the total sampling time. For each component (corresponding to overturning, slippage, obstacle, and slope respectively), the median, median absolute deviation (MAD), and scaled standard deviation estimates are calculated:

[0119]

[0120]

[0121]

[0122] in, It is the median. It is the median absolute deviation (MAD). It is a standardized standard deviation estimate. It is the first The o-th component of the information vector at time step, It is a constant that converts the absolute deviation of the median into an estimate of the standard deviation, derived from the statistical properties of the normal distribution;

[0123] Construct the Hampel inhibition factor to form a diagonal inhibition matrix:

[0124]

[0125]

[0126] in, It is the first The Hampel suppression factor of the o-th component of the innovation vector at time step 1. It is a diagonal suppression matrix. It is the outlier detection threshold coefficient. It is set through experiments. By constructing a multi-scenario test set in the simulation environment, including normal driving, occasional noise and sudden anomalies, the candidate values ​​are compared and evaluated. The state estimation accuracy, anomaly response speed and normal data preservation ability under each configuration are comprehensively evaluated. The optimal value that achieves the best balance between robustness and dynamic response is selected. The value range is [1.5, 3.5].

[0127] Based on diagonal suppression matrix We apply weighted suppression to the original information to obtain the weighted and suppressed robust information:

[0128]

[0129] in, It is a robust innovation after weighted suppression;

[0130] To adapt to slow environmental changes (such as wet roads in rainy weather and magnetic interference drift), the sample covariance matrix is ​​calculated based on the robust innovation after all weighted suppression:

[0131]

[0132] in, It is the sample covariance matrix. It is a transpose operation;

[0133] Through the sample covariance matrix Update observation noise covariance:

[0134]

[0135] in, It is the updated observation noise covariance. It is the minimum permissible observation noise covariance to prevent numerical instability. It is the prior estimation error covariance. It is a transpose operation;

[0136] Using the updated observation noise covariance Perform adaptive Kalman fusion and calculate the Kalman gain (scalar state → vector gain):

[0137]

[0138] in, It is the Kalman gain;

[0139] Update the overall risk estimate and error covariance:

[0140]

[0141]

[0142] in, This is the updated comprehensive risk assessment. It is the updated error covariance;

[0143] Updated comprehensive risk assessment Perform physical limiting:

[0144]

[0145] in, It is the risk estimate after the amplitude limit processing. [0,100] is the range defined by the comprehensive risk. 0 is no risk and 100 is the highest risk.

[0146] To reduce high-frequency jitter and improve control comfort, the risk estimate value after the previous time step's amplitude limiting processing is read. The optimal comprehensive risk estimate is obtained by performing a weighted root mean square estimation.

[0147]

[0148] in, It is the optimal comprehensive risk estimate, i.e., the comprehensive risk value. It is a weighting coefficient, set through an adaptive optimization mechanism. When a large change in environmental risk is detected, the weighting coefficient is increased to enhance the system's responsiveness to new information. When the environment tends to be stable, the weighting coefficient is decreased to suppress high-frequency fluctuations and improve control comfort. The value range is [0.3, 0.9].

[0149] By establishing an adaptive Kalman filter framework with comprehensive risk as the state variable, optimal fusion of multi-source sub-risks is achieved. Standard environmental calibration initialization ensures cold-start stability, and the Hampel robust suppression mechanism independently resists disturbances to each sub-risk channel, effectively isolating instantaneous sensor failures. Based on the innovation sample covariance, observation noise is dynamically updated, enabling the system to adapt to slow environmental changes. Combined with physical limiting and root mean square smoothing, the output risk value is ensured to have both engineering standardization and control smoothness. This fundamentally solves the problems of "poor robustness, rigid response, and susceptibility to interference" in traditional multi-source risk fusion. It significantly improves the reliability, adaptability, and user experience of the self-balancing scooter safety warning system in complex field scenarios, providing a high-confidence decision basis for graded intervention.

[0150] Furthermore, based on the comprehensive risk value, a tiered early warning and feedback learning mechanism is implemented. The magnitude is set, with three preset response levels (Level 1, Level 2, and Level 3), and different warning outputs and control interventions are adopted. Statistical analysis is used to set the threshold (collecting comprehensive risk estimates from various typical driving scenarios, constructing probability density distributions, and setting a Level 1 threshold). The secondary threshold is set so that 90% of the historical normal driving risk values ​​do not exceed this value. Set it so that 99% of the historical normal driving risk values ​​do not exceed this value, satisfying the following condition. , The value range is [30, 60]. Risk threshold with a value range of [60, 90] , ,and When the comprehensive risk value Less than the threshold If the overall risk value is low, a yellow LED light will flash, but no control intervention will be performed; the user will only be alerted. Greater than the threshold Less than the threshold If the system activates, the yellow LED light will flash rapidly, the controller will vibrate, and an app notification will be sent. The maximum speed will be limited, and the motor output torque will be reduced. This will be done when the overall risk value is within acceptable limits. Greater than the threshold If the red LED light stays on, a buzzer will sound and an APP pop-up window will appear, the drive motor output will be cut off, the electronic parking brake (EPB) will be activated, allowing only pushing operations and completely preventing driving. After each control intervention is executed, key information will be packaged and recorded.

[0151] The key information includes whether an impact event has occurred (determined in real time by the IMU), whether the user has lifted the intervention (recording whether the user manually lifted the speed limit or intervention mode in the APP or vehicle interface), the current road surface type (marking the current road surface category (such as "slippery mud", "asphalt", etc.)), and the actual risk result label (determined by posterior event analysis, whether the current decision successfully avoided the accident, marked as "safe" or "accident").

[0152] Key information is used to form standard experience tuples for reinforcement learning, which are used to train or fine-tune the DRL policy network. The standard experience tuples are fed into the DRL's experience replay pool for periodic retraining, thereby achieving continuous optimization of the sub-risk weight adjustment strategy.

[0153] A three-tiered risk response mechanism enables progressive safety intervention, from alerts to mandatory braking, balancing user experience with absolute safety under extreme conditions. Furthermore, key contexts and post-hoc results during the intervention process are constructed into structured experience tuples for online fine-tuning of the DRL strategy. This not only improves the accuracy of real-time protection but also establishes a closed-loop learning capability of "perception-decision-feedback-optimization," enabling the system to continuously improve its ability to identify and respond to complex risks over long-term use.

[0154] This embodiment also provides a computer device applicable to a multi-sensor-based safety warning method for a self-balancing vehicle in the field, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the multi-sensor-based safety warning method for a self-balancing vehicle in the field as proposed in the above embodiment.

[0155] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0156] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements a safety warning method for a multi-sensor-based field operation balance vehicle as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

Claims

1. A safety early warning method for a self-balancing scooter used in field operations based on multiple sensors, characterized in that: include, Collect multi-source sensor data and vibration signals from the self-balancing vehicle for field operations, preprocess the vibration signals, and calculate the terrain slope and confidence level based on the digital elevation model. Based on preprocessed vibration signals, a one-dimensional convolutional neural network is used to identify road surface types and output friction coefficients. Combining slope and confidence level, dynamic thresholds are obtained through reinforcement learning and curriculum learning strategies, and sub-risk values ​​are calculated. Based on the sub-risk values, an adaptive Kalman filter is used to fuse them to obtain a comprehensive risk value, and hierarchical early warning and feedback learning are performed according to the comprehensive risk value.

2. The method for safety early warning of a field operation self-balancing vehicle based on multiple sensors as described in claim 1, characterized in that: The process of collecting multi-source sensor data and vibration signals from the self-balancing vehicle for field operations and preprocessing the vibration signals refers to the main controller synchronously triggering source sensors at a fixed frequency to collect data during the operation of the self-balancing vehicle. This includes the vehicle's pitch angle, roll angle, and lateral acceleration; collecting the distance and height of obstacles ahead using a time-of-flight laser rangefinder; obtaining geographic coordinates (lat, lon), travel speed, and heading angle using a high-precision GPS module and electronic compass; acquiring the horizontal accuracy factor HDOP; collecting vertical vibration signals; and preprocessing the vertical vibration signals to obtain filtered signals.

3. The method for safety early warning of a field operation self-balancing vehicle based on multiple sensors as described in claim 2, characterized in that: The calculation of the forward terrain slope and confidence level based on the digital elevation model refers to obtaining the geographic coordinates of the target point ahead by using a geodetic forward calculation algorithm based on geographic coordinates (lat, lon), heading angle, and preset forward horizontal distance. Based on the digital elevation model, the geographic coordinates (lat, lon) and the geographic coordinates of the target point in front are... Perform bilinear interpolation to obtain the elevation of the current location. Elevation of the target point The slope is calculated, and the GPS horizontal positioning error is estimated by multiplying the horizontal accuracy factor HDOP with the preset user ranging error benchmark, and the slope confidence factor is calculated.

4. The safety early warning method for a field operation self-balancing vehicle based on multiple sensors as described in claim 3, characterized in that: The method involves using a pre-processed vibration signal and a one-dimensional convolutional neural network to identify the road surface type and output the friction coefficient. The filtered signal is used as input data and input into a pre-trained lightweight one-dimensional convolutional neural network model to obtain the prediction confidence of the road surface type and the friction coefficient.

5. A safety early warning method for a field operation self-balancing vehicle based on multiple sensors as described in claim 4, characterized in that: The combination of slope and confidence, and the acquisition of dynamic threshold through reinforcement learning and course learning strategies, refers to the process of assembling the collected data, slope, prediction confidence, slope confidence factor, and friction coefficient into a state vector of the deep reinforcement learning agent, inputting it into the deep reinforcement learning agent, and obtaining the dynamic safety threshold through the deep reinforcement learning agent. The deep reinforcement learning agent adopts a Double DQN architecture, comprising a master Q-network and a target Q-network. The network structure is a multilayer perceptron, and it is trained in stages in an offline simulation environment using a course learning strategy. During training, a reward enhancement mechanism is used to highlight critical security events. After each training stage, all interaction experiences are stored in an independent replay buffer, and forward propagation is performed. The master Q-network outputs the action Q-value and a dynamic threshold. The dynamic threshold is scaled and mapped using offset layers to obtain the final output value of the dynamic threshold, and a multi-objective loss function is calculated. Backpropagation and optimizer update of main Q network parameters are performed, and pruning and growth are carried out. The new sparse topology mask and corresponding weight matrix after pruning and growth are used as the main Q network parameters for the next training cycle. Simultaneously, the target Q network synchronizes the weight matrix of the main network through soft updates and hard copies the new sparse topology mask of the main network. When the average reward in the first phase of T consecutive evaluations reaches the success rate threshold for phase switching... When the training is complete, the state vector is input into the trained DRL agent, which only performs forward inference and outputs safety control parameters, i.e., dynamic thresholds, including low-risk overturning critical angle, high-risk overturning critical angle, high-risk slope threshold, and obstacle warning distance threshold. The corresponding weight matrix is ​​obtained through backpropagation training optimization under the updated sparse topological constraints.

6. The safety early warning method for a field operation self-balancing vehicle based on multiple sensors as described in claim 5, characterized in that: The calculation of sub-risk values ​​refers to calculating sub-risk values ​​based on dynamic thresholds, including overturning risk, slippage risk, obstacle risk, and slope risk.

7. A safety early warning method for a field operation self-balancing vehicle based on multiple sensors as described in claim 6, characterized in that: The method of obtaining a comprehensive risk value by fusion using adaptive Kalman filtering based on sub-risk values ​​refers to obtaining an initial comprehensive risk estimate and an initial error covariance during the operation of the self-balancing vehicle, and making a priori estimate of the current state based on the optimal comprehensive risk estimate and the corresponding error covariance at the previous moment. The sub-risk values ​​at the current moment are synchronously read as four noisy observations of the true comprehensive risk, forming an observation vector. An observation matrix is ​​then constructed to calculate the innovation, and robustness processing is performed to obtain the robust innovation after all weighted suppression. Robust innovations after weighted suppression Calculate the sample covariance matrix And update the observation noise covariance; Adaptive Kalman fusion is performed using the updated observation noise covariance to obtain an updated comprehensive risk estimate. The error covariance is used to estimate the updated overall risk. Physical limiting is applied, and a weighted root mean square estimation is performed to obtain the optimal comprehensive risk estimate. That is, the comprehensive risk value.

8. A safety early warning method for a field operation self-balancing vehicle based on multiple sensors as described in claim 7, characterized in that: The aforementioned implementation of tiered early warning and feedback learning based on comprehensive risk value refers to... The size is set, three response levels are preset, and different early warning outputs and control interventions are adopted. After each control intervention is executed, the key information is packaged and recorded, and the key information is combined into a reinforcement learning standard experience tuple for optimization training.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the safety early warning method for a field operation balance vehicle based on multiple sensors as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the multi-sensor-based field operation balance vehicle safety early warning method as described in any one of claims 1 to 8.

Images