A truck fatigue driving monitoring method and system based on multi-source data fusion

Abstract

The application discloses a truck fatigue driving monitoring method and system based on multi-source data fusion, and relates to the technical field of truck driving monitoring.The method comprises the following steps: obtaining vehicle related original data from various types of sensing terminals on the expressway, forming a unified time-space data set, constructing a vehicle trajectory reconstruction model with a gantry as a strong constraint, reconstructing and fusing the vehicle trajectory, generating a continuous vehicle driving trajectory, extracting multi-dimensional fatigue features, constructing a fatigue fusion feature vector, generating a fatigue driving risk level through a fatigue risk determination algorithm, classifying and processing the fatigue driving risk through a hierarchical linkage intervention strategy combining online early warning and offline disposal, collecting disposal feedback data, and iteratively optimizing the fatigue risk determination algorithm and the hierarchical linkage intervention strategy.The application can effectively curb traffic accidents caused by fatigue driving and provides an efficient and reliable solution for expressway traffic safety control.

Description

Technical Field

[0001] This invention relates to the field of truck driving monitoring technology, and in particular to a method and system for monitoring truck fatigue driving based on multi-source data fusion. Background Technology

[0002] Fatigue driving is a major contributing factor to serious traffic accidents on highways. Current fatigue driving monitoring methods still have many significant limitations and are insufficient to meet the safety supervision needs of highway driving: First, the data source is singular, with monitoring relying excessively on GPS (Global Positioning System) or single video path, resulting in incomplete coverage and numerous blind spots; second, the judgment logic is simplistic, using only continuous driving time as the criterion for fatigue driving, ignoring the effectiveness of actual rest and real-world driving scenarios such as multiple drivers alternating; third, intervention is delayed and passive, with most monitoring being post-event alarm modes, lacking effective pre-event warning methods and tiered handling mechanisms; fourth, the system is isolated, with monitoring, early warning, enforcement, and feedback links disconnected from each other, making it difficult to form a complete closed-loop management system.

[0003] Although current fatigue driving monitoring mainly relies on single data from GPS or ETC (Electronic Toll Collection) systems, these methods expose a series of specific problems that urgently need to be addressed in the complex environment of highways. Firstly, there are data silos and distortion issues. GPS signals are prone to drift or interruption in tunnels and mountainous areas, while simple ETC gantry data only provides discrete point information, failing to accurately depict the continuous driving state between gantries or identify abnormal lateral deviations such as swerving. Secondly, the judgment dimensions are singular and lagging. Existing technologies mostly rely on static thresholds of "4 hours of driving, 20 minutes of rest" for judgment, without considering the interference of environmental factors such as congestion, accidents, and weather on driving time, easily leading to false alarms. Simultaneously, there is a disconnect in identity recognition. The monitoring system cannot effectively identify dual-driver situations where drivers switch mid-journey, thus failing to properly reset the driver's fatigue state, leading to misjudgments of compliant vehicles. Furthermore, the intervention mechanism lacks a closed-loop design. Traditional reminder methods such as SMS have low reach rates, and the entire monitoring and intervention process lacks post-intervention effect evaluation and a corresponding model self-evolution mechanism.

[0004] Based on the aforementioned issues, how to construct a high-precision, comprehensive, interconnected, and closed-loop intelligent monitoring and proactive intervention system for fatigued driving, and realize full-process management of fatigued driving behavior of trucks on highways, has become an important research and practice direction for highway traffic safety supervision. Summary of the Invention

[0005] This invention provides a method for monitoring truck fatigue driving based on multi-source data fusion, comprising: Step S1: Obtain vehicle-related raw data from various sensing terminals on the highway, correlate them, and form a raw vehicle dataset; Step S2: Clean the original vehicle dataset of abnormal data and unify its spatiotemporal reference to form a unified spatiotemporal dataset. Step S3: Based on the unified spatiotemporal dataset, construct a vehicle trajectory reconstruction model with gantry as a strong constraint, reconstruct and fuse the vehicle trajectory to generate a continuous vehicle driving trajectory; Step S4: Based on the unified spatiotemporal dataset and the vehicle's continuous driving trajectory, extract multi-dimensional fatigue features and construct a fatigue fusion feature vector; Step S5: Based on the fatigue fusion feature vector, generate the fatigue driving risk level through the fatigue risk determination algorithm; Step S6: Based on the fatigue driving risk level, fatigue driving risk is classified and handled in a graded manner through a graded linkage intervention strategy that combines online early warning and offline handling. Step S7: Collect feedback data on the handling of fatigue driving risks, and iteratively optimize the fatigue risk assessment algorithm and the graded linkage intervention strategy.

[0006] The truck fatigue driving monitoring method based on multi-source data fusion described above includes the following sub-steps: acquiring vehicle-related raw data from various sensing terminals on highways and associating it to form a raw vehicle dataset. Step S11: Obtain raw vehicle-related data from various sensing terminals on the highway, including vehicle passage transaction data, trajectory data, vehicle passing visual data, and roadside facility event data. Step S12: Using the unique vehicle identifier in the vehicle passage transaction data as the core association key, associate the original data related to the same vehicle to form a single vehicle's original dataset.

[0007] The truck fatigue driving monitoring method based on multi-source data fusion, as described above, includes the following sub-steps: cleaning abnormal data from the original vehicle dataset and unifying its spatiotemporal reference to form a unified spatiotemporal dataset. Step S21: Based on preset business rules, perform abnormal data cleaning on the original vehicle dataset to obtain the vehicle dataset; Step S22: Using the timestamp of ETC gantry transactions as the time reference and the highway lane-level linear reference system as the spatial reference, perform spatiotemporal calibration on the vehicle dataset to form a unified spatiotemporal dataset for vehicles.

[0008] The above-described method for monitoring truck fatigue driving based on multi-source data fusion includes the following sub-steps: Based on a unified spatiotemporal dataset, a vehicle trajectory reconstruction model with a gantry as a strong constraint is constructed to reconstruct and fuse the vehicle trajectory, generating a continuous vehicle driving trajectory. Step S31: Using two consecutive ETC gantry events as the start and end points, divide the vehicle's journey into multiple analysis segments and extract the trajectory data of the corresponding analysis segments from the spatiotemporal unified dataset. Step S32: Define the vehicle motion state vector, construct a state prediction model based on extended Kalman filter, predict the trajectory data of each analysis segment time by time, and generate a priori state estimation vector. Step S33: Construct a hierarchical observation and update mechanism with the gantry as a strong constraint, calibrate the prior state estimation vector, reconstruct and fuse the trajectory of each analysis segment to form a continuous vehicle driving trajectory.

[0009] The above-described method for monitoring truck fatigue driving based on multi-source data fusion includes the following sub-steps: extracting multi-dimensional fatigue features and constructing a fatigue fusion feature vector based on a unified spatiotemporal dataset and the vehicle's continuous driving trajectory. Step S41: Based on the continuous driving trajectory of the vehicle, extract the fatigue features of the vehicle driving dimension; Step S42: Based on the unified spatiotemporal dataset and the vehicle's continuous driving trajectory, extract fatigue features in terms of driver identity and driving duration. Step S43: Construct a fatigue fusion feature vector based on fatigue features in the dimensions of vehicle driving, driver identity, and driving time.

[0010] The truck fatigue driving monitoring method based on multi-source data fusion, as described above, includes the following sub-steps for generating a fatigue driving risk level based on a fatigue fusion feature vector and a fatigue risk determination algorithm: Step S51: Divide the dataset based on the fatigue fusion feature vectors of historical vehicles and build the training environment; Step S52: Perform automatic hyperparameter optimization of random forest based on Bayesian optimization to obtain the optimal hyperparameter combination; Step S53: Based on the optimal hyperparameter combination, construct and train a cost-sensitive random forest model; Step S54: Input the fatigue fusion feature vector into the cost-sensitive random forest model to output the fatigue driving risk level and risk probability.

[0011] This invention also provides a truck fatigue driving monitoring system based on multi-source data fusion, comprising: The vehicle raw dataset generation module acquires vehicle-related raw data from various sensing terminals on the highway, correlates them, and forms a vehicle raw dataset. The spatiotemporal unified dataset generation module cleans the original vehicle dataset of abnormal data and unifies its spatiotemporal reference to form a spatiotemporal unified dataset. The vehicle continuous driving trajectory generation module, based on a unified spatiotemporal dataset, constructs a vehicle trajectory reconstruction model with gantry as a strong constraint, reconstructs and fuses vehicle trajectories, and generates a continuous vehicle driving trajectory. The fatigue fusion feature vector construction module extracts multi-dimensional fatigue features based on a unified spatiotemporal dataset and the continuous driving trajectory of vehicles, and constructs a fatigue fusion feature vector. The fatigue driving risk level generation module generates a fatigue driving risk level based on fatigue fusion feature vectors and a fatigue risk determination algorithm. The fatigue driving risk management module, based on the fatigue driving risk level, uses a tiered and coordinated intervention strategy that combines online early warning with offline handling to classify and manage fatigue driving risks. The optimization module collects feedback data on the handling of fatigue driving risks and iteratively optimizes the fatigue risk assessment algorithm and the graded linkage intervention strategy.

[0012] As described above, a truck fatigue driving monitoring system based on multi-source data fusion includes a vehicle raw dataset generation module, which specifically comprises: The vehicle-related raw data generation submodule acquires vehicle-related raw data from various sensing terminals on the highway, including vehicle passage transaction flow data, trajectory data, vehicle passing visual data, and roadside facility event data. The single-vehicle raw dataset generation submodule uses the unique vehicle identifier in the vehicle passage transaction data as the core association key to associate the vehicle-related raw data of the same vehicle to form a single-vehicle raw dataset.

[0013] The truck fatigue driving monitoring system based on multi-source data fusion described above includes a spatiotemporal unified dataset formation module, which specifically comprises: The vehicle dataset acquisition submodule performs abnormal data cleaning on the original vehicle dataset based on preset business rules to acquire the vehicle dataset. The spatiotemporal unified dataset formation submodule uses the timestamp of ETC gantry transactions as the time reference and the highway lane-level linear reference system as the spatial reference to perform spatiotemporal calibration on the vehicle dataset, thus forming a spatiotemporal unified dataset for vehicles.

[0014] As described above, a truck fatigue driving monitoring system based on multi-source data fusion includes a vehicle continuous driving trajectory generation module, which specifically comprises: The segment division submodule divides the vehicle's journey into multiple analysis segments, taking the two consecutive ETC gantry events as the start and end points, and extracts the trajectory data of the corresponding analysis segments from the unified spatiotemporal dataset. The prior state estimation vector generation submodule defines the vehicle motion state vector, constructs a state prediction model based on extended Kalman filter, performs time-by-time prediction on the trajectory data of each analysis segment, and generates the prior state estimation vector. The driving trajectory reconstruction and fusion submodule constructs a hierarchical observation and update mechanism with the gantry as a strong constraint, calibrates the prior state estimation vector, and reconstructs and fuses the trajectories of each analysis segment to form a continuous driving trajectory of the vehicle.

[0015] As described above, a truck fatigue driving monitoring system based on multi-source data fusion includes a fatigue fusion feature vector construction module, which specifically comprises: The vehicle driving dimension fatigue feature extraction submodule extracts fatigue features based on the vehicle's continuous driving trajectory. The driver fatigue feature extraction submodule extracts fatigue features based on a unified spatiotemporal dataset and continuous vehicle driving trajectories, focusing on driver identity and driving duration. The fatigue fusion feature vector construction submodule constructs fatigue fusion feature vectors based on fatigue features in the vehicle driving dimension, driver identity, and driving duration dimension.

[0016] The fatigue driving monitoring system for trucks based on multi-source data fusion, as described above, includes a fatigue driving risk level generation module that specifically comprises: The dataset is divided into sub-modules based on the fatigue fusion feature vectors of historical vehicles, and a training environment is constructed. The optimal hyperparameter combination acquisition submodule automatically optimizes the hyperparameters of a random forest based on Bayesian optimization to obtain the optimal hyperparameter combination. The cost-sensitive random forest model training submodule constructs and trains a cost-sensitive random forest model based on the optimal hyperparameter combination. The fatigue driving risk level output submodule inputs the fatigue fusion feature vector into the cost-sensitive random forest model and outputs the fatigue driving risk level and risk probability.

[0017] The beneficial effects achieved by this invention are as follows: This invention effectively solves the industry pain points of existing highway fatigue driving monitoring, such as numerous monitoring blind spots, poor data quality, and rigid judgment logic. It significantly reduces the false alarm rate under congested conditions, lowering it by more than 30% compared to traditional technologies, while maintaining a judgment accuracy rate of over 90% even under extreme monitoring conditions, achieving broad coverage and high precision in fatigue driving status monitoring. Furthermore, it constructs a complete hierarchical linkage intervention closed loop, enabling precise handling of vehicles with different risk situations. It achieves a second-level response from online risk alerts to offline interception and control, effectively curbing traffic accidents caused by fatigue driving. This creates a high technical barrier in the field of intelligent traffic management, providing an efficient and reliable solution for highway traffic safety control. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a flowchart of a truck fatigue driving monitoring method based on multi-source data fusion provided in Embodiment 1 of this application; Figure 2 This is a schematic diagram of a truck fatigue driving monitoring system based on multi-source data fusion provided in Embodiment 2 of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 like Figure 1 As shown in Embodiment 1 of this application, a method for monitoring truck fatigue driving based on multi-source data fusion is provided. The method includes the following steps: Step S1: Obtain vehicle-related raw data from various sensing terminals on the highway, correlate them, and form a raw vehicle dataset; Furthermore, acquiring raw vehicle-related data from various sensing terminals on the highway and correlating it to form a raw vehicle dataset includes the following sub-steps: Step S11: Obtain raw vehicle-related data from various sensing terminals on the highway, including vehicle passage transaction data, trajectory data, vehicle passing visual data, and roadside facility event data. Specifically, the system connects to the real-time data interface of the highway ETC gantry system. After a vehicle triggers an ETC gantry transaction, it collects real-time vehicle passage transaction data, including the vehicle's unique identifier, millisecond-level timestamp, gantry geolocation code, coordinates, station number, and vehicle direction of travel. Through the data transmission link of the vehicle's Beidou / GPS terminal, it batches the original satellite trajectory positioning point data stream of the corresponding vehicle according to a preset time window (e.g., every 10 seconds). This data stream includes data items such as timestamp, latitude and longitude, speed, and license plate. When the ETC gantry detects a vehicle transaction... Upon receiving the signal, the high-definition imaging equipment mounted on the gantry is immediately triggered to collect visual data of the corresponding vehicle, including high-definition images of the vehicle passing through and continuous video streams. The high-definition images of the vehicle passing through include at least a clear image of the front of the vehicle and a clear image of the driver's cab, and include a timestamp and a gantry code watermark. The continuous video streams are continuous video streams of the vehicle before and after passing through the gantry within a preset time period. The system also connects to the event recording system of roadside facilities such as service area entrances and exits and toll stations to collect records of vehicle entry and exit events at these locations, including vehicle identification, event timestamp, event type, facility type, and location code.

[0022] Step S12: Using the unique vehicle identifier in the vehicle passage transaction data as the core association key, associate the original data related to the same vehicle to form the original vehicle dataset for a single vehicle. Specifically, using the unique vehicle identifier in the vehicle passage transaction data as the core association key and the timestamp of each data item in the vehicle-related raw data as the auxiliary key, the vehicle-related raw data such as the vehicle passage transaction data, trajectory data, vehicle passing visual data and roadside facility event data of the same vehicle within a preset time window are associated and bound to form a vehicle raw dataset with the unique vehicle identifier of the corresponding vehicle as the core index.

[0023] Step S2: Clean the original vehicle dataset of abnormal data and unify its spatiotemporal reference to form a unified spatiotemporal dataset. Furthermore, the original vehicle dataset undergoes anomaly cleaning, and its spatiotemporal reference is standardized to form a unified spatiotemporal dataset. This process includes the following sub-steps: Step S21: Based on preset business rules, perform abnormal data cleaning on the original vehicle dataset to obtain the vehicle dataset; Specifically, a dedicated business rule base is developed based on the highway freight scenario. Core rules include reasonable speed range, consistency of time logic, standardization of data format, and reasonableness of spatial location. Based on the dedicated business rule base, abnormal data is processed by category for the original vehicle dataset. Invalid transaction records in the vehicle passage transaction flow data that lack unique vehicle identification, timestamps, or gantry coding errors are removed. Trajectory drift points, invalid points, and abnormal points in trajectory data are filtered out. Invalid event records in roadside facility event data that fail to match license plates, have ambiguous event type labels, or have incorrect facility coding are removed. Valid data that conforms to the business rules is retained to form the vehicle dataset.

[0024] Step S22: Using the timestamp of ETC gantry transactions as the time reference and the highway lane-level linear reference system as the spatial reference, perform spatiotemporal calibration on the vehicle dataset to form a unified spatiotemporal dataset for vehicles. Specifically, using the precise UTC time of ETC gantry transactions as the authoritative time source, the timestamps of all data in the vehicle dataset are synchronized and calibrated; at the same time, a high-precision map matching algorithm is called to map the latitude and longitude information in the trajectory data to the lane-level linear reference system of the highway, unifying the spatiotemporal reference of all data in the vehicle dataset, and obtaining a unified spatiotemporal dataset of vehicles.

[0025] Step S3: Based on the unified spatiotemporal dataset, construct a vehicle trajectory reconstruction model with gantry as a strong constraint, reconstruct and fuse the vehicle trajectory to generate a continuous vehicle driving trajectory. Furthermore, based on a unified spatiotemporal dataset, a vehicle trajectory reconstruction model with gantry as a strong constraint is constructed to reconstruct and fuse vehicle trajectories, generating continuous vehicle driving trajectories. This process includes the following sub-steps: Step S31: Using two consecutive ETC gantry events as the start and end points, divide the vehicle's journey into multiple analysis segments and extract the trajectory data of the corresponding analysis segments from the spatiotemporal unified dataset. Specifically, taking two consecutive ETC gantry events of a single vehicle as the start and end points, the vehicle's complete journey on the highway is divided into multiple independent analysis segments. The extraction of ETC gantry events involves sorting consecutive ETC transaction records of a single vehicle in ascending order by transaction timestamp, extracting the core information from two adjacent gantries, and forming the basic parameters of the analysis segment, including time parameters. With spatial parameters , For the timestamp of the vehicle entering the gate. For the timestamp of the vehicle leaving the gate, The vehicle's entry gate location coordinates, The geographic coordinates of the vehicle exiting the gate; the time parameters of each analysis segment. To constrain this, the rules for extracting trajectory data are set as follows: Extracting all satellite trajectory positioning points that satisfy the time constraint truncation rules for the corresponding analysis segment from the unified spatiotemporal dataset, which represents the vehicle's spatial parameters. Continuous satellite trajectory positioning data is used to extract satellite trajectory positioning points, which are then arranged in ascending order by timestamp to form a continuous sequence of trajectory positioning points. Based on the speed limit thresholds for each section of the highway and combined with the actual driving characteristics of trucks, a drift point determination threshold is set. Based on the distance difference and time difference between two adjacent trajectory positioning points in the sequence, a drift point determination threshold is established. Calculate the instantaneous vehicle speed and remove outliers where the instantaneous speed exceeds a threshold. The instantaneous speed of the vehicle. The distance difference between two adjacent trajectory positioning points. It represents the time difference between two adjacent trajectory positioning points.

[0026] Step S32: Define the vehicle motion state vector, construct a state prediction model based on extended Kalman filter, predict the trajectory data of each analysis segment time by time, and generate a priori state estimation vector. Specifically, based on the station number corresponding to the ETC gantry at the starting point of each analysis segment and the initial driving speed of the vehicle, the optimal state estimation vector at time k-1 is obtained. Perform initialization assignment, setting the vehicle's motion state vector at time k as follows: ,in, Let k be the longitudinal mileage position (station) of the vehicle in the lane-level linear reference frame at time k. Let k be the instantaneous speed of the vehicle at time k. Let K be the instantaneous acceleration of the vehicle at time k; based on the fundamental laws of vehicle kinematics, construct the state prediction equation for the vehicle from time k-1 to time k. The trajectory positioning points within each analysis segment are predicted at fixed time steps, filling in the state gaps between trajectory positioning points within each analysis segment. Let be the prior state estimation vector from time k-1 to time k. Here is the state transition matrix. Let be the optimal state estimation vector at time k-1. For process noise to satisfy a normal distribution N(0,Q), Q is the process noise covariance matrix.

[0027] Step S33: Construct a hierarchical observation update mechanism with the gantry as a strong constraint, calibrate the prior state estimation vector, reconstruct and fuse the trajectory of each analysis segment to form a continuous vehicle driving trajectory; Specifically, based on the observation accuracy and reliability of the ETC gantry and the BeiDou / GPS terminals mounted on the vehicles, the ETC gantry is set as a strong observation constraint, while the trajectory data collected by the BeiDou / GPS terminals mounted on the vehicles is set as a weak observation constraint, and the observation noise covariance R is dynamically adjusted. When conventional satellite trajectory positioning point data is received, the observation noise covariance is set based on the inherent observation uncertainty of the data collected by the BeiDou / GPS terminals. To represent the typical error level of the trajectory data, a certain degree of smoothing fluctuation is allowed. When a vehicle triggers an ETC gantry transaction, the station number of the ETC gantry in the lane-level linear reference system is obtained. Set the observation noise covariance According to the Kalman gain formula Calculate the Kalman gain, where, The Kalman gain at time k, Let $\mathbf{k-1}$ be the prediction error covariance of the prior state from time k-1 to time k. For the measurement matrix, To observe the noise covariance (when the observed object is an ETC gantry) When the observed object is trajectory data ); based on Kalman gain Vehicle condition correction formula For the prior state estimation vector The following corrections were made: Let be the optimal state estimation vector at time k. Let be the prior state estimation vector from time k-1 to time k. The Kalman gain at time k, Real-time observation value (when the observation object is an ETC gantry) , The gantry number used to trigger an ETC gantry transaction When the observed object is trajectory data , (Location information after mapping of trajectory positioning points) The measurement matrix is ​​used; based on the latitude and longitude information of the trajectory data, the lateral offset of all trajectory positioning points in each analysis segment is obtained. The optimal state estimation vectors at all times in the analysis segment are sorted in ascending order by timestamp. Combined with the lateral offset, a lane-level continuous fusion trajectory is generated. It is then matched with the high-precision road network map of the highway to form a high-confidence continuous vehicle driving trajectory that strictly fits the road network.

[0028] Step S4: Based on the unified spatiotemporal dataset and the vehicle's continuous driving trajectory, extract multi-dimensional fatigue features and construct a fatigue fusion feature vector; Furthermore, based on the unified spatiotemporal dataset and the continuous driving trajectory of the vehicle, multi-dimensional fatigue features are extracted, and a fatigue fusion feature vector is constructed, including the following sub-steps: Step S41: Based on the continuous driving trajectory of the vehicle, extract the fatigue features of the vehicle driving dimension; Specifically, the lateral offset and longitudinal mileage position are extracted from the vehicle's continuous driving trajectory, and then analyzed using the vehicle displacement fluctuation formula. Calculate the vehicle displacement volatility, where S is the vehicle displacement volatility. This represents the change in lateral offset of a vehicle relative to the road centerline within a unit of time period. The vertical mileage increment within a unit of time period. The compensation coefficient is derived from speed (based on the speed limit characteristics of trucks on highways); based on the continuous driving trajectory of vehicles and combined with the width of highway lanes, the number of lateral crossings of vehicles per unit time is counted (excluding necessary lane changes caused by external factors such as road construction and congestion), and the lane change frequency L of vehicles per unit time is calculated; based on the instantaneous speed in the continuous driving trajectory of vehicles, a congestion judgment threshold is set, the speed data of congested road sections in the continuous driving trajectory of vehicles are removed, the speed data of uniform speed road sections per unit time is counted, and the speed fluctuation variance var is calculated for the speed sequence per unit time.

[0029] Step S42: Based on the unified spatiotemporal dataset and the vehicle's continuous driving trajectory, extract fatigue features in terms of driver identity and driving duration. Specifically, the system extracts vehicle-passing visual data from two consecutive ETC gantries in a spatiotemporal unified dataset. The driver's facial region is cropped, and a convolutional neural network is used to extract the driver's facial feature vector. The cosine similarity algorithm is then used to calculate the similarity between the driver's facial feature vectors at the two gantries, and this similarity is used as a score for identity continuity. Based on the recognition accuracy, a similarity threshold is set. When the similarity is lower than the threshold, it is determined that the driver has been changed, and an identity reset feature is generated. The image of the driver's cabin area in the vehicle pass-through visual data is located, and the target is detected by a head detection model. The actual number of people on duty in the driver's cabin is counted, and an on-duty personnel identification feature is generated. , represents the actual number of people tested. When the system detects multi-person collaborative driving, it automatically activates the collaborative driving mode and adjusts the weight of continuous driving time. From the vehicle's continuous driving trajectory, it identifies the cumulative driving time during which the driver has driven continuously without effective rest, and records this as the effective driving time. The determination of an effective rest period involves extracting vehicle entry and exit event records from the roadside facility event data in the unified spatiotemporal dataset. The duration of each vehicle's stay in the service area is calculated based on the difference in timestamps between the service area entrance and exit. A threshold of 20 minutes is set for determining an effective rest period; if the stay is greater than or equal to 20 minutes, it is considered an effective rest, and the effective driving time is recorded. Reset to zero, and after the vehicle resumes driving, the effective driving time will be recorded. The driving time is recalculated. If the rest time is less than 20 minutes, it is considered an invalid rest. The valid driving time is calculated as follows. Instead of resetting the timer, the travel time for subsequent road segments will continue to accumulate.

[0030] Step S43: Construct a fatigue fusion feature vector based on fatigue features in the dimensions of vehicle driving, driver identity, and driving time; Specifically, the vehicle displacement fluctuation rate S, lane change frequency L, and speed fluctuation variance var in the vehicle driving dimension are normalized, and the effective driving time in the driver identity and driving duration dimensions is normalized. Quantification was performed hourly; vehicle displacement volatility S, lane change frequency L, speed volatility variance var, and identity continuity score were analyzed. On-duty personnel identification characteristics With effective driving time Standardize and integrate the data to construct a fatigue fusion feature vector. .

[0031] Step S5: Based on the fatigue fusion feature vector, generate the fatigue driving risk level through the fatigue risk determination algorithm; Furthermore, based on the fatigue fusion feature vector, the fatigue driving risk level is generated through a fatigue risk assessment algorithm, including the following sub-steps: Step S51: Divide the dataset based on the fatigue fusion feature vectors of historical vehicles and build the training environment; Specifically, the process involves obtaining multiple batches of fatigue fusion feature vectors X from historical vehicles, along with their corresponding real fatigue state labels. These real fatigue state labels are generated based on accident records, manual annotation results, and traffic enforcement records. The multiple batches of fatigue fusion feature vectors X from historical vehicles and their corresponding real fatigue state labels together form a sample set, which is then divided into a training set, a validation set, and a test set according to a preset ratio. A random forest model and a Bayesian optimizer are initialized, and the search space for hyperparameters is set, including the range of values ​​for the number of decision trees N, the maximum depth D, and the maximum feature selection M per tree.

[0032] Step S52: Perform automatic hyperparameter optimization of random forest based on Bayesian optimization to obtain the optimal hyperparameter combination; Specifically, the performance evaluation metric of the random forest model on the validation set is defined as the objective function. Using a Bayesian optimization algorithm, the objective function is maximized through iterative searching within a defined hyperparameter search space. In each iteration, the Bayesian optimizer, based on a Gaussian process regression model, predicts and selects the next hyperparameter combination most likely to improve performance based on historical hyperparameter combinations and their corresponding objective function values. This process is repeated multiple times until the objective function no longer improves, at which point the iteration stops, and the number of decision trees at that point is output. Maximum depth Feature selection quantity Generate the optimal hyperparameter combination .

[0033] Step S53: Based on the optimal hyperparameter combination, construct and train a cost-sensitive random forest model; Specifically, a cost matrix is ​​defined based on the safety requirements of fatigue driving warning scenarios. Normal driving is labeled as category 0, and fatigued driving is labeled as category 1. Cost matrix. The elements in are, This represents the cost of determining whether driving is actually normal and is predicted to be normal. This represents the cost of a false alarm that indicates normal driving but is predicted as fatigued driving. This represents the cost of underreporting actual fatigue driving when it is predicted as normal driving. The cost of determining whether a driver is actually fatigued and is predicted to be fatigued is defined; based on the risk priority of fatigued driving warning scenarios, cost weights are assigned to each element of the cost matrix, which satisfy the following conditions: Based on optimal hyperparameter combination The random forest model is initialized. During the training process, when each decision tree splits a node, all training samples contained in the node to be split are obtained to form a node sample set. Based on the true class label of each sample, the node sample set is divided into two subsets: a subset of normal driving samples with a true class of 0 and a subset of fatigued driving samples with a true class of 1. Based on the cost matrix C, each sample is assigned a weight value representing its misclassification cost. For a normal driving sample with a true class of 0, its weight value is equal to the false alarm cost of misclassifying it as fatigued driving. For a fatigued driving sample whose true category is 1, its weight is equal to the cost of underreporting it by misclassifying it as normal driving. For each candidate splitting feature and its corresponding splitting threshold, the weighted Gini coefficient of the child nodes generated after the split is calculated. The splitting feature and splitting threshold that minimize the overall weighted Gini coefficient after the split are selected as the optimal splitting scheme for the current decision tree node. The random forest model is iteratively trained using the training set and its performance is verified using the validation set. The core verification indicators are a false negative rate lower than a preset false negative threshold and a false positive rate lower than a preset false positive threshold. When the performance of the random forest model meets the core verification indicators, the model training is terminated, and the cost-sensitive random forest model that determines fatigue risk is output.

[0034] Step S54: Input the fatigue fusion feature vector into the cost-sensitive random forest model to output the fatigue driving risk level and risk probability; Specifically, the real-time acquired vehicle fatigue fusion feature vector X is input into the cost-sensitive random forest model. The model calculates the probability that the current state belongs to normal driving (category 0) by integrating the prediction results of all decision trees. The probability of being classified as fatigued driving (Category 1) ;Will As the risk probability output, it is denoted as Based on the cost-sensitive principle, the formula for minimizing the expected cost is used. Determine the decision categories, where, Decision category (0 indicates normal, 1 indicates fatigue). Candidate decision category (0 or 1). For true categories (0 or 1). The categories calculated for the model are The probability, This represents the decision cost corresponding to the cost matrix; based on the decision category and risk probability, combined with a preset risk threshold, it is mapped to a three-level risk level. and When the risk level is below the low-risk threshold, it is considered low-risk. , Higher than the low risk threshold and When the risk level is below the high-risk threshold, it is classified as medium-risk. and If the risk level is higher than the high-risk threshold, it is considered high-risk, and the output is {risk probability:} Risk level: High / Medium / Low.

[0035] Step S6: Based on the fatigue driving risk level, fatigue driving risk is classified and handled in a graded manner through a graded linkage intervention strategy that combines online early warning and offline handling. Specifically, when the risk level is low, a warning instruction is sent to the roadside information board control system. The instruction includes the vehicle's unique identification number, license plate, current location, and direction of travel. The roadside information board control system automatically matches the nearest roadside variable message sign based on the vehicle's current location and displays standardized warning information. When the risk level is medium, the communication module is invoked to automatically dial the driver's pre-registered phone number for manual reminders and guide the vehicle to the nearest service area for rest. When the risk level is high, a deployment instruction is pushed to the command platform of the relevant traffic management department. The instruction includes full-dimensional information such as the vehicle's license plate, unique identification number, current location, direction of travel, and probability of fatigue risk. Based on the vehicle's current location and direction of travel, the command platform sets up deployment interception points at the nearest toll station, service area, or mainline checkpoint. At the same time, vehicle information is pushed to the staff at the deployment points, and traffic police and road administration personnel are dispatched to intercept the vehicle on-site and handle the fatigue driving behavior.

[0036] Step S7: Collect feedback data on the handling of fatigue driving risks, and iteratively optimize the fatigue risk assessment algorithm and the graded linkage intervention strategy. Specifically, the actual data after intervention, such as road interception results, accident data, and violation records, are summarized to form a feedback dataset. The random forest model in the fatigue risk assessment algorithm is iteratively trained and its parameters are optimized. Historical graded intervention data are statistically analyzed to evaluate the effectiveness of intervention measures for each risk level. The intervention strategies for each risk level are then targeted and optimized, and the graded response strategies corresponding to low / medium / high risks are dynamically adjusted.

[0037] Example 2 like Figure 2 As shown, Embodiment 2 of this application provides a truck fatigue driving monitoring system based on multi-source data fusion, including: The vehicle raw dataset generation module 21 acquires vehicle-related raw data from various sensing terminals on the highway, correlates them, and forms a vehicle raw dataset. Furthermore, the vehicle raw dataset generation module 21 includes the following sub-modules: The vehicle-related raw data generation submodule acquires vehicle-related raw data from various sensing terminals on the highway, including vehicle passage transaction flow data, trajectory data, vehicle passing visual data, and roadside facility event data. The single-vehicle raw dataset generation submodule uses the unique vehicle identifier in the vehicle passage transaction data as the core association key to associate the vehicle-related raw data of the same vehicle to form a single-vehicle raw dataset. The spatiotemporal unified dataset generation module 22 cleans the original vehicle dataset of abnormal data and unifies its spatiotemporal reference to form a spatiotemporal unified dataset. Furthermore, the spatiotemporal unified dataset formation module 22 includes the following sub-modules: The vehicle dataset acquisition submodule performs abnormal data cleaning on the original vehicle dataset based on preset business rules to acquire the vehicle dataset. The spatiotemporal unified dataset formation submodule uses the timestamp of ETC gantry transactions as the time reference and the highway lane-level linear reference system as the spatial reference to perform spatiotemporal calibration on the vehicle dataset, forming a spatiotemporal unified dataset for vehicles. The vehicle continuous driving trajectory generation module 23, based on the unified spatiotemporal dataset, constructs a vehicle trajectory reconstruction model with the gantry as a strong constraint, reconstructs and merges the vehicle trajectory, and generates a continuous vehicle driving trajectory. Furthermore, the vehicle continuous driving trajectory generation module 23 includes the following sub-modules: The segment division submodule divides the vehicle's journey into multiple analysis segments, taking the two consecutive ETC gantry events as the start and end points, and extracts the trajectory data of the corresponding analysis segments from the unified spatiotemporal dataset. The prior state estimation vector generation submodule defines the vehicle motion state vector, constructs a state prediction model based on extended Kalman filter, performs time-by-time prediction on the trajectory data of each analysis segment, and generates the prior state estimation vector. The driving trajectory reconstruction and fusion submodule constructs a hierarchical observation and update mechanism with the gantry as a strong constraint, calibrates the prior state estimation vector, and reconstructs and fuses the trajectories of each analysis segment to form a continuous driving trajectory of the vehicle. The fatigue fusion feature vector construction module 24 extracts multi-dimensional fatigue features based on a unified spatiotemporal dataset and the continuous driving trajectory of the vehicle, and constructs a fatigue fusion feature vector. Furthermore, the fatigue fusion feature vector construction module 24 includes the following sub-modules: The vehicle driving dimension fatigue feature extraction submodule extracts fatigue features based on the vehicle's continuous driving trajectory. The driver fatigue feature extraction submodule extracts fatigue features based on a unified spatiotemporal dataset and continuous vehicle driving trajectories, focusing on driver identity and driving duration. The fatigue fusion feature vector construction submodule constructs fatigue fusion feature vectors based on fatigue features in the vehicle driving dimension, driver identity, and driving duration dimension. The fatigue driving risk level generation module 25 generates a fatigue driving risk level based on the fatigue fusion feature vector and through a fatigue risk judgment algorithm. Furthermore, the fatigue driving risk level generation module 25 includes the following sub-modules: The dataset is divided into sub-modules based on the fatigue fusion feature vectors of historical vehicles, and a training environment is constructed. The optimal hyperparameter combination acquisition submodule automatically optimizes the hyperparameters of a random forest based on Bayesian optimization to obtain the optimal hyperparameter combination. The cost-sensitive random forest model training submodule constructs and trains a cost-sensitive random forest model based on the optimal hyperparameter combination. The fatigue driving risk level output submodule inputs the fatigue fusion feature vector into the cost-sensitive random forest model and outputs the fatigue driving risk level and risk probability. The fatigue driving risk handling module 26, based on the fatigue driving risk level, uses a graded linkage intervention strategy that combines online early warning and offline handling to classify and handle fatigue driving risks. Optimization module 27 collects feedback data on the handling of fatigue driving risks and iteratively optimizes the fatigue risk assessment algorithm and the graded linkage intervention strategy. Corresponding to the above embodiments, the present invention provides a computer storage medium, including: at least one memory and at least one processor; The memory is used to store one or more program instructions; A processor is used to run one or more program instructions to execute a truck fatigue driving monitoring method based on multi-source data fusion.

[0038] Corresponding to the above embodiments, this embodiment of the invention provides a computer-readable storage medium containing one or more program instructions, which are used by a processor to provide a method for monitoring truck fatigue driving based on multi-source data fusion.

[0039] The embodiments disclosed in this invention provide a computer-readable storage medium storing computer program instructions. When the computer program instructions are executed on a computer, the computer executes the above-described method for monitoring truck fatigue driving based on multi-source data fusion.

[0040] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0041] The various methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods.

[0042] The storage medium can be memory, such as volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0043] Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0044] Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM).

[0045] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0046] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using a combination of hardware and software. When applied as software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0047] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring truck fatigue driving based on multi-source data fusion, characterized in that, include: Step S1: Obtain vehicle-related raw data from various sensing terminals on the highway, correlate them, and form a raw vehicle dataset; Step S2: Clean the original vehicle dataset of abnormal data and unify its spatiotemporal reference to form a unified spatiotemporal dataset. Step S3: Based on the unified spatiotemporal dataset, construct a vehicle trajectory reconstruction model with gantry as a strong constraint, reconstruct and fuse the vehicle trajectory to generate a continuous vehicle driving trajectory. Step S4: Based on the unified spatiotemporal dataset and the vehicle's continuous driving trajectory, extract multi-dimensional fatigue features and construct a fatigue fusion feature vector; Step S5: Based on the fatigue fusion feature vector, generate the fatigue driving risk level through the fatigue risk determination algorithm; Step S6: Based on the fatigue driving risk level, fatigue driving risk is classified and handled in a graded manner through a graded linkage intervention strategy that combines online early warning and offline handling. Step S7: Collect feedback data on the handling of fatigue driving risks, and iteratively optimize the fatigue risk assessment algorithm and the graded linkage intervention strategy.

2. The method for monitoring truck fatigue driving based on multi-source data fusion as described in claim 1, characterized in that, The process of acquiring raw vehicle-related data from various sensing terminals on highways and correlating it to form a raw vehicle dataset includes the following sub-steps: Step S11: Obtain raw vehicle-related data from various sensing terminals on the highway, including vehicle passage transaction data, trajectory data, vehicle passing visual data, and roadside facility event data. Step S12: Using the unique vehicle identifier in the vehicle passage transaction data as the core association key, associate the original data related to the same vehicle to form a single vehicle's original dataset.

3. The method for monitoring truck fatigue driving based on multi-source data fusion as described in claim 1, characterized in that, Based on a unified spatiotemporal dataset, a vehicle trajectory reconstruction model with gantry as a strong constraint is constructed to reconstruct and fuse vehicle trajectories, generating continuous vehicle driving trajectories. The process includes the following sub-steps: Step S31: Using two consecutive ETC gantry events as the start and end points, divide the vehicle's journey into multiple analysis segments and extract the trajectory data of the corresponding analysis segments from the spatiotemporal unified dataset. Step S32: Define the vehicle motion state vector, construct a state prediction model based on extended Kalman filter, predict the trajectory data of each analysis segment time by time, and generate a priori state estimation vector. Step S33: Construct a hierarchical observation and update mechanism with the gantry as a strong constraint, calibrate the prior state estimation vector, reconstruct and fuse the trajectory of each analysis segment to form a continuous vehicle driving trajectory.

4. The method for monitoring truck fatigue driving based on multi-source data fusion as described in claim 1, characterized in that, Based on a unified spatiotemporal dataset and continuous vehicle trajectories, multi-dimensional fatigue features are extracted, and a fatigue fusion feature vector is constructed, including the following sub-steps: Step S41: Based on the continuous driving trajectory of the vehicle, extract the fatigue features of the vehicle driving dimension; Step S42: Based on the unified spatiotemporal dataset and the vehicle's continuous driving trajectory, extract fatigue features in terms of driver identity and driving duration. Step S43: Construct a fatigue fusion feature vector based on fatigue features in the dimensions of vehicle driving, driver identity, and driving time.

5. The method for monitoring truck fatigue driving based on multi-source data fusion as described in claim 1, characterized in that, Based on fatigue fusion feature vectors, the fatigue driving risk level is generated through a fatigue risk assessment algorithm, including the following sub-steps: Step S51: Divide the dataset based on the fatigue fusion feature vectors of historical vehicles and build the training environment; Step S52: Perform automatic hyperparameter optimization of random forest based on Bayesian optimization to obtain the optimal hyperparameter combination; Step S53: Based on the optimal hyperparameter combination, construct and train a cost-sensitive random forest model; Step S54: Input the fatigue fusion feature vector into the cost-sensitive random forest model to output the fatigue driving risk level and risk probability.

6. A truck fatigue driving monitoring system based on multi-source data fusion, characterized in that, include: The vehicle raw dataset generation module acquires vehicle-related raw data from various sensing terminals on the highway, correlates them, and forms a vehicle raw dataset. The spatiotemporal unified dataset generation module cleans the original vehicle dataset of abnormal data and unifies its spatiotemporal reference to form a spatiotemporal unified dataset. The vehicle continuous driving trajectory generation module, based on a unified spatiotemporal dataset, constructs a vehicle trajectory reconstruction model with gantry as a strong constraint, reconstructs and fuses vehicle trajectories, and generates a continuous vehicle driving trajectory. The fatigue fusion feature vector construction module extracts multi-dimensional fatigue features based on a unified spatiotemporal dataset and the continuous driving trajectory of vehicles, and constructs a fatigue fusion feature vector. The fatigue driving risk level generation module generates a fatigue driving risk level based on fatigue fusion feature vectors and a fatigue risk determination algorithm. The fatigue driving risk management module, based on the fatigue driving risk level, uses a tiered and coordinated intervention strategy that combines online early warning with offline handling to classify and manage fatigue driving risks. The optimization module collects feedback data on the handling of fatigue driving risks and iteratively optimizes the fatigue risk assessment algorithm and the graded linkage intervention strategy.

7. A truck fatigue driving monitoring system based on multi-source data fusion as described in claim 6, characterized in that, The vehicle raw dataset generation module specifically includes: The vehicle-related raw data generation submodule acquires vehicle-related raw data from various sensing terminals on the highway, including vehicle passage transaction flow data, trajectory data, vehicle passing visual data, and roadside facility event data. The single-vehicle raw dataset generation submodule uses the unique vehicle identifier in the vehicle passage transaction data as the core association key to associate the vehicle-related raw data of the same vehicle to form a single-vehicle raw dataset.

8. A truck fatigue driving monitoring system based on multi-source data fusion as described in claim 6, characterized in that, The vehicle continuous driving trajectory generation module specifically includes: The segment division submodule divides the vehicle's journey into multiple analysis segments, taking the two consecutive ETC gantry events as the start and end points, and extracts the trajectory data of the corresponding analysis segments from the unified spatiotemporal dataset. The prior state estimation vector generation submodule defines the vehicle motion state vector, constructs a state prediction model based on extended Kalman filter, performs time-by-time prediction on the trajectory data of each analysis segment, and generates the prior state estimation vector. The driving trajectory reconstruction and fusion submodule constructs a hierarchical observation and update mechanism with the gantry as a strong constraint, calibrates the prior state estimation vector, and reconstructs and fuses the trajectories of each analysis segment to form a continuous driving trajectory of the vehicle.

9. A truck fatigue driving monitoring system based on multi-source data fusion as described in claim 6, characterized in that, The fatigue fusion feature vector construction module specifically includes: The vehicle driving dimension fatigue feature extraction submodule extracts fatigue features based on the vehicle's continuous driving trajectory. The driver fatigue feature extraction submodule extracts fatigue features based on a unified spatiotemporal dataset and continuous vehicle driving trajectory, taking into account driver identity and driving duration. The fatigue fusion feature vector construction submodule constructs fatigue fusion feature vectors based on fatigue features in the vehicle driving dimension, driver identity, and driving duration dimension.

10. A truck fatigue driving monitoring system based on multi-source data fusion as described in claim 6, characterized in that, The fatigue driving risk level generation module specifically includes: The dataset is divided into sub-modules based on the fatigue fusion feature vectors of historical vehicles, and a training environment is constructed. The optimal hyperparameter combination acquisition submodule automatically optimizes the hyperparameters of a random forest based on Bayesian optimization to obtain the optimal hyperparameter combination. The cost-sensitive random forest model training submodule constructs and trains a cost-sensitive random forest model based on the optimal hyperparameter combination. The fatigue driving risk level output submodule inputs the fatigue fusion feature vector into the cost-sensitive random forest model and outputs the fatigue driving risk level and risk probability.

Images