Vehicle monitoring method and system based on multi-source data fusion

Abstract

The invention provides a vehicle monitoring method and system based on multi-source data fusion, and belongs to the technical field of vehicle monitoring, and the method comprises the steps: obtaining multiple types of data, the information sources of the multiple types of data are different; determining target data corresponding to the multiple types of data based on the multiple types of data; fusing the target data corresponding to the multiple types of data to obtain comprehensive data; and monitoring the vehicle based on the comprehensive data to obtain a vehicle monitoring result. According to the vehicle monitoring method and system based on multi-source data fusion, the accuracy and reliability of vehicle monitoring can be improved.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of vehicle monitoring, and more specifically, to a vehicle monitoring method and system based on multi-source data fusion. Background Art

[0002] With the continuous increase in traffic volume, vehicle monitoring is of great significance for traffic management, safety and security. Traditional vehicle monitoring methods often rely on a single data source, such as camera image recognition or geomagnetic sensor detection. However, these single data source monitoring methods have limitations. For example, they affect the accuracy of vehicle behavior and feature recognition or monitoring.

[0003] Therefore, how to achieve more accurate and reliable vehicle monitoring has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present disclosure is to provide a vehicle monitoring method and system based on multi-source data fusion to improve the accuracy and reliability of vehicle monitoring.

[0005] A first aspect of the embodiments of the present disclosure provides a vehicle monitoring method based on multi-source data fusion, comprising: Acquire multiple types of data, where the multiple types of data come from different sources; Determine target data corresponding to the multiple types of data based on the multiple types of data; fusing target data corresponding to the multiple types of data to obtain comprehensive data; The vehicle is monitored based on the comprehensive data to obtain a vehicle monitoring result.

[0006] A second aspect of the embodiments of the present disclosure provides a vehicle monitoring system based on multi-source data fusion, including: A data acquisition module, used to acquire multiple types of data, where the multiple types of data have different information sources; A target data determination module, used to determine target data corresponding to the multiple types of data based on the multiple types of data; A fusion module, used for fusing target data corresponding to the multiple types of data to obtain comprehensive data; The monitoring module is used to monitor the vehicle based on the comprehensive data to obtain vehicle monitoring results.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned vehicle monitoring method based on multi-source data fusion when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned vehicle monitoring method based on multi-source data fusion are implemented.

[0009] The vehicle monitoring method and system based on multi-source data fusion provided by the embodiments of the present disclosure have the following beneficial effects: By acquiring multiple types of data from different information sources, the present disclosure can more comprehensively capture various information related to the vehicle, including but not limited to the vehicle's driving status, traffic environment, driver behavior, etc., providing a rich data basis for vehicle monitoring. Secondly, the present disclosure can obtain more accurate and reliable comprehensive data by fusing the target data corresponding to multiple types of data. The above data fusion process can effectively reduce the errors or limitations caused by a single data source and improve the accuracy and credibility of the monitoring results. Finally, monitoring the vehicle based on the comprehensive data can obtain more comprehensive and in-depth vehicle monitoring results. Therefore, the present disclosure can improve the accuracy and reliability of vehicle monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram of a process flow of a vehicle monitoring method based on multi-source data fusion provided in an embodiment of the present disclosure; Figure 2 A structural block diagram of a vehicle monitoring system based on multi-source data fusion provided by an embodiment of the present disclosure; Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0013] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0014] Please refer to Figure 1 , Figure 1 A schematic flow chart of a vehicle monitoring method based on multi-source data fusion provided in an embodiment of the present disclosure, the method comprising: S101: Acquire multiple types of data, where the information sources of the multiple types of data are different.

[0015] In this embodiment, information about the vehicle and the surrounding environment status can be collected from multiple dimensions or in multiple ways. The above data complement each other and can provide a more comprehensive perspective to understand the vehicle's situation, while avoiding the limitations of a single information source.

[0016] The multiple types of data may include static information, real-time vehicle information, and real-time road and environment information. Static information may include roads, facilities, and equipment. The collected data for static information is shown in Table 1: Table 1 Collection data of static information

[0017] The real-time vehicle information may include the real-time vehicle trajectory, the real-time abnormal events of the vehicle, and the vehicle license plate recognition information at the toll station. The data collection for the real-time vehicle information is shown in Table 2.

[0018] Table 2 Collection data of vehicle real-time information

[0019] The road environment information may include emergency information, meteorological environment information, lane status information and road operation status information. Data collection of the road environment information is performed, as shown in Table 3.

[0020] Table 3 Collection data of road environment information

[0021] The above data can be obtained through cameras, geomagnetic sensors, millimeter radar waves, and communication between roadside units and on-board units.

[0022] S102: Determine target data corresponding to the multiple categories of data based on the multiple categories of data.

[0023] In this embodiment, if one of the multiple types of data is image data, it is necessary to adjust the clarity of the image data to obtain target image data; If one of the multiple types of data is numerical data, it is necessary to determine the target numerical data based on data cleaning and data anomaly screening.

[0024] Considering that image data is easily affected by lighting, environment, shooting angle, etc., it is necessary to process the image, such as image enhancement, image weakening, etc., to adjust the image clarity, so as to obtain the target image data.

[0025] Taking into account that numerical data may have outliers or individual extreme cases, the numerical data is cleaned or outliers are removed to obtain numerical data with more reference value as the target numerical data.

[0026] Therefore, compared with the original multi-class data, the target data removes redundant and abnormal information.

[0027] S103: Fusing target data corresponding to multiple types of data to obtain comprehensive data.

[0028] In this embodiment, fusion is a process of integrating target data from multiple different sources that are interrelated and have a supplementary description of the same monitored object (vehicle) according to a fusion method or fusion strategy, so that they work together to form an information set that can more comprehensively and accurately reflect the vehicle status.

[0029] Comprehensive data is a new data set obtained after fusion processing, which can include multi-dimensional information such as vehicle appearance, location, speed, operating status, driving intention, etc. Compared with single target data, it can provide a more complete and reliable basis for vehicle monitoring.

[0030] There are many types of target data, and the factors that affect them will also be different. Therefore, weights can be set to adjust the emphasis of each target data to obtain fused data that is more in line with the actual situation.

[0031] S104: Monitor the vehicle based on the comprehensive data to obtain a vehicle monitoring result.

[0032] In this embodiment, vehicle monitoring is a process of observing, analyzing and judging the various performances of vehicles on the road using algorithms or models based on the information contained in the comprehensive data, in order to detect whether the vehicle has abnormal behavior and to understand the vehicle dynamics for traffic management.

[0033] The vehicle monitoring results are conclusions about various aspects of the vehicle, including but not limited to whether the vehicle complies with traffic regulations, whether the vehicle itself is in a healthy operating state, and the prediction of the vehicle's next driving direction. They are the final output of the entire monitoring process and provide a direct basis for traffic control, vehicle maintenance, etc.

[0034] For example, the real-time speed of the vehicle is compared with the speed limit of the road section. The comprehensive data contains the real-time speed information of the vehicle, and the system pre-stores the speed limit values ​​of each road section. For example, on a section of a city trunk road, the speed limit is 60 kilometers per hour. If the comprehensive data shows that the current speed of the vehicle is 70 kilometers per hour, the comparison can determine that the vehicle is speeding, and the result of "speeding" is recorded in the vehicle monitoring results, which can be used by the traffic police for law enforcement later.

[0035] Analyze the continuity and rationality of vehicle trajectories. Use the continuous position information of vehicles in the comprehensive data to determine whether the vehicle trajectory conforms to the normal driving logic. For example, at an intersection, the vehicle should normally drive according to the lane guidance. If the comprehensive data shows that the vehicle suddenly deviates from the designated lane and drives across the lane without a reasonable lane change signal, it can be determined that the vehicle has violated the lane change behavior, and the monitoring result is recorded as "illegal lane change" to facilitate timely correction of traffic violations.

[0036] From the above, it can be concluded that the present disclosure can more comprehensively capture various information related to the vehicle, including but not limited to the vehicle's driving status, traffic environment, driver behavior, etc., by acquiring multiple types of data from different information sources, thereby providing a rich data basis for vehicle monitoring. Secondly, the present disclosure can obtain more accurate and reliable comprehensive data by fusing the target data corresponding to multiple types of data. The above data fusion process can effectively reduce the errors or limitations caused by a single data source and improve the accuracy and credibility of the monitoring results. Finally, monitoring the vehicle based on the comprehensive data can obtain more comprehensive and in-depth vehicle monitoring results. Therefore, the present disclosure can improve the accuracy and reliability of vehicle monitoring.

[0037] In one embodiment of the present disclosure, the multiple types of data include image data and numerical data; Determine multiple categories of target data based on multiple categories of data, including: Evaluate the image data based on the image quality evaluation method to obtain the image data quality; Evaluate the numerical data based on the numerical data evaluation method to obtain the quality of the numerical data; Determine the target image preprocessing strategy based on the image data quality; Determine the target numerical preprocessing strategy based on the quality of numerical data; Processing the image data based on the target image preprocessing strategy to obtain the first type of target data; The numerical data is processed based on the target numerical preprocessing strategy to obtain the second type of target data.

[0038] In this embodiment, the image data may be data in the form of pictures of the vehicle appearance, driving scenes, etc. captured by a camera, and the numerical data may be data presented in digital form such as vehicle speed values, position coordinate values, and various quantitative parameters measured by sensors.

[0039] Image quality assessment methods are used to measure the performance of images in terms of clarity, contrast, noise level, and integrity. The above methods can objectively determine whether the image meets the requirements of subsequent processing and analysis, such as whether the image is clear enough to identify vehicle features.

[0040] The image quality assessment method may be a pixel-based method, and the specific steps are as follows: First, calculate the mean of the sum of squares of the differences between the corresponding pixel values ​​of the standard image and the image to be evaluated, i.e., the mean square error (MSE). Assume that the pixel value of the standard image is , the pixel value of the image to be evaluated is , the calculation formula of MSE is: ,in and are the width and height of the image respectively. The smaller the MSE value, the closer the image quality is to the standard image, and the better the quality is. The standard image is the standard of the image to be evaluated.

[0041] Second, calculate the peak signal-to-noise ratio (PSNR) based on MSE calculation, which reflects the ratio of the maximum signal power to the noise power of the image. The higher the PSNR value, the better the image quality. Its formula is ,in is the maximum possible value of the image pixel.

[0042] Determine the target image preprocessing strategy based on image data quality, including: If the image data quality is less than a first image quality threshold, image enhancement is used as a target image preprocessing strategy; If the image data quality is greater than or equal to the first image quality threshold and less than the second image quality threshold, the image preprocessing strategy is not executed; If the image data quality is greater than a second image quality threshold, image weakening is used as a target image preprocessing strategy; The first image quality threshold is smaller than the second image quality threshold.

[0043] In this embodiment, when the image data quality is within the range of the first image quality threshold and the second image quality threshold, the image data quality is relatively standard and does not require further processing. When the image data quality is less than the first image quality threshold, the image brightness is relatively dark, and image enhancement is required. When the image data quality is greater than or equal to the second image quality threshold, the image brightness is somewhat exposed, and image weakening is required to make the overall image look softer. After the above target image preprocessing strategy is used, the first type of target data can be obtained.

[0044] Numerical data evaluation methods are used to judge the quality dimensions of data, such as accuracy, reliability, and validity. For example, the quality of numerical data can be measured by judging whether the vehicle speed value is within a reasonable range, whether the value measured by the sensor has abnormal fluctuations, etc.

[0045] In this embodiment, the numerical data may be clustered based on a clustering algorithm to obtain a first cluster of data and a second cluster of data; In response to the first cluster of data having a much larger amount than the second cluster of data: If the amount of the second cluster of data is less than or equal to the first amount threshold, the second cluster of data is determined to be abnormal data and needs to be removed; If the amount of the second cluster data is greater than the first amount threshold, the second cluster data is retained for further analysis.

[0046] The clustering algorithm can well identify the existence of outliers, and the abnormal data needs to be eliminated. After the above, the second type of target data is obtained.

[0047] It can be concluded from the above that this embodiment achieves accurate evaluation and processing of multiple types of data by comprehensively considering the quality of image data and numerical data. First, a special evaluation method is used to obtain the quality information of image data and numerical data respectively, and then the most suitable preprocessing strategy is customized based on this quality information. It can not only significantly improve the accuracy and availability of data, but also effectively avoid subsequent analysis errors caused by data quality problems.

[0048] In one embodiment of the present disclosure, the vehicle monitoring method based on multi-source data fusion further includes: If the target data is real-time vehicle data, the vehicle features are used as target features; If the target data is real-time road data, the road condition characteristics are used as target characteristics; If the target data is real-time environmental data, the meteorological features are used as target features; Determine the target fusion strategy based on the target features of multiple types of data; Based on the target fusion strategy, multiple types of target data are fused to obtain comprehensive data.

[0049] In one embodiment of the present disclosure, the target fusion strategy includes a vehicle fusion strategy, a road condition fusion strategy, and a weather fusion strategy; Determine the target fusion strategy based on the target features of multiple types of data, including: In response to the target feature being a vehicle feature, selecting a vehicle fusion strategy for fusion; In response to the target feature being a road condition feature, selecting a road condition fusion strategy for fusion; In response to the target feature being a meteorological feature, a meteorological fusion strategy is selected for fusion.

[0050] In one embodiment of the present disclosure, multiple types of target data are fused based on a target fusion strategy to obtain comprehensive data, including: Determine a vehicle fusion result of the vehicle fusion strategy; Determine the traffic condition fusion result of the traffic condition fusion strategy; Determine the meteorological fusion results of the meteorological fusion strategy; The vehicle fusion results, road condition fusion results and weather fusion results are weighted and calculated to obtain comprehensive data.

[0051] In this embodiment, the first type of target data or the second type of target data may include vehicle real-time data, road real-time data and environment real-time data respectively.

[0052] Vehicle real-time data can reflect the current status of the vehicle, such as the real-time location of the vehicle, which can be longitude and latitude coordinates; real-time speed of the vehicle, etc. Vehicle features are features extracted from the real-time data of the vehicle, such as the location features, speed features, and vehicle model features of the vehicle.

[0053] Real-time road data refers to data related to the road status obtained in real time, such as real-time road traffic flow, that is, the number of vehicles passing a certain road section per unit time; lane occupancy, that is, whether there are vehicles running on each lane, whether there are obstacles occupying each lane, etc.; road construction status, that is, whether construction is in progress, the scope of the construction section, etc. Road condition characteristics are elements that can characterize the characteristics of the road status, such as the degree of traffic congestion and road passability.

[0054] Environmental real-time information is data that reflects the vehicle's surrounding environment in real time, such as meteorological data, ambient noise level, and light intensity. Meteorological characteristics can characterize current weather conditions.

[0055] The above information can be obtained by a camera, a sensor, or some information database. Therefore, the above data includes both image data and numerical data, but they are all ultimately converted into numerical data for subsequent processing. The numerical data may also include vectors, matrices, etc.

[0056] The vehicle fusion strategy can be carried out using the weighted average method, which assigns corresponding weights to different types of vehicles, and then performs weighted summation to obtain the vehicle fusion result.

[0057] The road condition fusion strategy can be integrated using the Kalman filter method. The Kalman filter method is an optimal estimation algorithm based on the state space model. It can continuously estimate and correct through state equations and observation equations, through prediction and updating. It can effectively process road condition data containing noise and finally obtain road condition fusion results.

[0058] For example, when monitoring highway conditions, the average speed and traffic volume of a road section can be used as state variables. Real-time speed and traffic volume data are obtained through sensors installed on the road as observation values, and the Kalman filter method is used to fuse these data to predict and update the road condition of the road section, effectively remove noise interference in the data, and obtain more accurate road condition information.

[0059] The meteorological fusion strategy can be a wavelet transform fusion algorithm. Wavelet transform can decompose meteorological data into different frequency scales to obtain wavelet coefficients at different scales. After wavelet transforming meteorological data from different data sources, the wavelet coefficients are fused according to the fusion rules, and then the fused wavelet coefficients are reconstructed into fused meteorological data through inverse wavelet transform. In this way, the features of meteorological data can be fused at different scales, the detailed information of the data can be retained, and finally the meteorological fusion result can be obtained.

[0060] For example, for the satellite cloud image data and the humidity data of the ground meteorological station, firstly, wavelet transform is performed on them respectively to obtain wavelet coefficients at different scales and positions. Then, weights are determined according to factors such as the reliability of the satellite cloud image and the ground meteorological station data, and the wavelet coefficients are fused. Finally, humidity data that integrates the satellite cloud image and the ground meteorological station information is obtained through inverse wavelet transform, which can more accurately reflect the atmospheric humidity conditions.

[0061] Perform weighted calculation on vehicle fusion results, road condition fusion results, and weather fusion results to obtain comprehensive data; Assume that the vehicle fusion result is , the traffic fusion result is The meteorological fusion result is ; First, determine the impact coefficient.

[0062] The vehicle driving stability coefficient is , the value range is , and its calculation formula is:

[0063] in, For the vehicle The speed of the time interval, is the total number of time intervals, is the total time, is the maximum speed of the vehicle during this period.

[0064] The traffic congestion coefficient is , the value range is , the traffic congestion index is , the calculation formula is .

[0065] The weather severity coefficient is , the value range is , visibility is , the standard visibility is , the calculation formula is .

[0066] Second, dynamically adjust weights.

[0067] The weight of the vehicle fusion result is ; The weight of the traffic fusion result is ; The weight of the meteorological fusion result is .

[0068] Third, the calculation formula for comprehensive data is:

[0069] in, Indicates Vehicle fusion results, Indicates Traffic fusion results, Indicates The meteorological fusion result.

[0070] It can be concluded from the above that this embodiment can make full use of multi-source data, improve the accuracy and comprehensiveness of vehicle monitoring, and provide strong support for traffic management, safety warning, etc. At the same time, this embodiment adopts vehicle fusion strategy, road condition fusion strategy and meteorological fusion strategy in a targeted manner to obtain key information in three dimensions: vehicle, road and environment. Afterwards, by weighted calculation of the above three fusion results, not only the advantages of multi-source data are fully integrated, but also the accuracy and comprehensiveness of the comprehensive data are ensured.

[0071] In one embodiment of the present disclosure, a vehicle is monitored based on comprehensive data to obtain a vehicle monitoring result, including: The comprehensive data is input into a vehicle monitoring model to determine a vehicle monitoring result, wherein the vehicle monitoring model is trained based on historical vehicle comprehensive data.

[0072] In this embodiment, the vehicle monitoring model can be a machine learning model architecture, which can analyze and judge the vehicle status and behavior based on the input comprehensive data, and can output vehicle monitoring results. The model has learned a large number of characteristic patterns of vehicles in normal and abnormal states. The historical vehicle comprehensive data is the collected and accumulated comprehensive data, which records the operating conditions of vehicles at different times and in different scenarios, including various forms of normal driving, illegal driving, failures, congestion, and driving in bad weather, which can enable the vehicle monitoring model to learn the rules between the historical vehicle comprehensive data and the corresponding historical vehicle monitoring results.

[0073] Specifically, the construction of the vehicle monitoring model can be as follows: First, the model architecture is selected. The architecture that combines the Long Short-Term Memory (LSTM) network and the Convolutional Neural Network (CNN) is used.

[0074] LSTM is used to process time series data during vehicle driving, such as changes in vehicle speed and acceleration over time. It can capture long-term dependencies in the data and remember the vehicle's operating trends over a period of time, which is crucial for judging changes in the vehicle's dynamic performance. For example, by continuously monitoring changes in vehicle speed, LSTM can determine whether the vehicle has sudden acceleration or deceleration anomalies, and whether such anomalies persist, and then infer whether the vehicle's driving status is normal.

[0075] CNN is used to process image data and extract spatial features from multi-source data. Vehicle appearance images and road scene images collected by cameras can be used to identify vehicle type, vehicle body damage, road signs, traffic light status and other information through CNN. At the same time, for some data with spatial distribution characteristics in the fused data (such as traffic flow distribution on different road sections, etc.), CNN can also effectively extract key features and convert them into valuable information for vehicle monitoring.

[0076] Second, the model input layer.

[0077] Design multiple input branches to adapt to comprehensive data in multiple dimensions.

[0078] Third, the model hidden layer.

[0079] Connect LSTM and CNN modules: The LSTM layer consists of multiple LSTM units, and the appropriate number of hidden units is set according to the length and complexity of the input time series data to fully learn the dynamic characteristics of vehicle operation.

[0080] The CNN module contains multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer extracts local features of images and spatial data through convolution kernels of different sizes. The pooling layer is used to reduce the data dimension and reduce the amount of calculation. The fully connected layer further integrates the extracted features and outputs a high-dimensional feature vector.

[0081] After the LSTM and CNN modules, a fusion layer is set to concatenate the feature vectors output by the two modules and fuse them through a fully connected layer, so that the model can comprehensively utilize time series and spatial feature information to comprehensively judge the vehicle status.

[0082] Fourth, the model output layer.

[0083] Design multiple output nodes corresponding to different vehicle monitoring results: A node outputs the judgment result of whether the vehicle is speeding, with a value of 0 (not speeding) or 1 (speeding).

[0084] Another node is used to determine the degree of traffic congestion and output 0 (unblocked), 1 (mild congestion), 2 (moderate congestion), and 3 (severe congestion).

[0085] There is also a node that determines the degree of impact of weather conditions on vehicle driving, and outputs 0 (no impact), 1 (slight impact, such as light rain, light breeze), 2 (moderate impact, such as moderate rain, strong wind), and 3 (severe impact, such as heavy rain, heavy snow).

[0086] Fifth, model training.

[0087] Data preparation: Collect a large amount of historical vehicle comprehensive data, including vehicle operation data under different models, different road conditions, and various weather conditions. Label the data, for example, clarify whether the vehicle corresponding to each piece of data is speeding, the level of road congestion, the degree of weather impact, etc., to ensure the diversity and balance of the data and avoid too much or too little data of a certain type affecting the model learning effect.

[0088] Divide the data into training set, validation set and test set according to a certain ratio, such as 70% for training, 15% for validation and 15% for testing.

[0089] Training process: Select a suitable loss function. For example, the cross entropy loss function can be used for multi-classification problems, and the mean square error loss function can be used for continuous value prediction (such as vehicle speed prediction for speeding judgment).

[0090] Use optimization algorithms, such as stochastic gradient descent and its variants Adagrad, Adadelta, Adam, etc., to iteratively update model parameters, and continuously adjust parameters such as weights and biases in LSTM and CNN to gradually reduce the loss of the model on the training set. At the same time, monitor the performance of the model on the validation set to prevent overfitting.

[0091] Model evaluation and optimization: Evaluate the trained model on the test set, calculate the accuracy, recall, F1 value and other indicators, and evaluate the accuracy of the model's judgment on different vehicle monitoring results. Based on the evaluation results, further optimize the model, such as adjusting the model architecture, increasing the amount of data, adjusting hyperparameters, etc., until the model reaches satisfactory performance indicators.

[0092] Sixth, determine the vehicle monitoring model.

[0093] It can be concluded from the above that this embodiment can realize accurate and rapid monitoring of vehicle status by using the vehicle monitoring model to learn and analyze historical vehicle comprehensive data. It not only improves the efficiency and accuracy of vehicle monitoring, but also helps to timely discover potential vehicle problems and prevent safety accidents. At the same time, the monitoring process is highly automated and intelligent, which reduces manual intervention and reduces monitoring costs.

[0094] In one embodiment of the present disclosure, the vehicle monitoring method based on multi-source data fusion further includes: Based on the vehicle monitoring results, determine whether the vehicle has violated traffic regulations or engaged in dangerous behavior.

[0095] In this embodiment, illegal behavior refers to the operation behavior of the vehicle driver that violates traffic regulations, which commonly include speeding, running red lights, illegal lane changes, and illegal parking. The above behaviors disrupt traffic order and are likely to cause traffic accidents.

[0096] Although dangerous behavior does not necessarily violate traffic regulations directly, it is a behavior that puts the vehicle in a potentially dangerous state due to improper driver operation or sudden vehicle failure under certain road and weather conditions. For example, driving at high speed on slippery roads, continuing to drive when the vehicle's brake system fails, and driving due to fatigue causing abnormal vehicle driving trajectories, etc., all of which increase the risk of accidents.

[0097] This embodiment uses the monitoring results output by the vehicle monitoring model to deeply analyze the comprehensive data, which includes information such as the vehicle's current operating status, road conditions, and meteorological environment. Based on traffic regulations and safe driving common sense, it is determined whether the vehicle has violated traffic regulations or has fallen into a dangerous behavior pattern that may cause an accident due to various factors, so that timely intervention measures can be taken to ensure road traffic safety.

[0098] From the above, it can be concluded that this embodiment can quickly identify illegal driving (such as speeding, running red lights) or dangerous driving behaviors (such as sudden braking, frequent lane changes) by real-time monitoring of vehicle status and comparing it with preset rules, providing timely and accurate law enforcement basis for traffic management departments. It not only helps maintain traffic order, but also effectively prevents traffic accidents and ensures public safety.

[0099] Corresponding to the vehicle monitoring method based on multi-source data fusion in the above embodiment, Figure 2 This is a structural block diagram of a vehicle monitoring system based on multi-source data fusion provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 2 The vehicle monitoring system 20 based on multi-source data fusion includes: a data acquisition module 21, a target data determination module 22, a fusion module 23 and a monitoring module 24.

[0100] The data acquisition module 21 is used to acquire multiple types of data, and the information sources of the multiple types of data are different; A target data determination module 22, configured to determine target data corresponding to the multiple types of data based on the multiple types of data; A fusion module 23 is used to fuse target data corresponding to multiple types of data to obtain comprehensive data; The monitoring module 24 is used to monitor the vehicle based on the comprehensive data to obtain the vehicle monitoring result.

[0101] In one embodiment of the present disclosure, the multiple types of data include image data and numerical data; The target data determination module 22 is specifically used for: Evaluate the image data based on the image quality evaluation method to obtain the image data quality; Evaluate the numerical data based on the numerical data evaluation method to obtain the quality of the numerical data; Determine the target image preprocessing strategy based on the image data quality; Determine the target numerical preprocessing strategy based on the quality of numerical data; Processing the image data based on the target image preprocessing strategy to obtain the first type of target data; The numerical data is processed based on the target numerical preprocessing strategy to obtain the second type of target data.

[0102] In one embodiment of the present disclosure, the vehicle monitoring system 20 based on multi-source data fusion further includes: a fusion strategy determination module; A fusion strategy determination module, for taking vehicle features as target features if the target data is real-time vehicle data; If the target data is real-time road data, the road condition characteristics are used as target characteristics; If the target data is real-time environmental data, the meteorological features are used as target features; Determine the target fusion strategy based on the target features of multiple types of data; Based on the target fusion strategy, multiple types of target data are fused to obtain comprehensive data.

[0103] In one embodiment of the present disclosure, the target fusion strategy includes a vehicle fusion strategy, a road condition fusion strategy, and a weather fusion strategy; The fusion strategy determination module is specifically used for: In response to the target feature being a vehicle feature, selecting a vehicle fusion strategy for fusion; In response to the target feature being a road condition feature, selecting a road condition fusion strategy for fusion; In response to the target feature being a meteorological feature, a meteorological fusion strategy is selected for fusion.

[0104] In one embodiment of the present disclosure, the fusion strategy determination module is further configured to: Determine vehicle fusion results of the vehicle fusion strategy; Determine the traffic condition fusion result of the traffic condition fusion strategy; Determine the meteorological fusion results of the meteorological fusion strategy; The vehicle fusion results, road condition fusion results and weather fusion results are weighted and calculated to obtain comprehensive data.

[0105] In one embodiment of the present disclosure, the monitoring module 24 is specifically configured to: The comprehensive data is input into a vehicle monitoring model to determine a vehicle monitoring result, wherein the vehicle monitoring model is trained based on historical vehicle comprehensive data.

[0106] In one embodiment of the present disclosure, the vehicle monitoring system 20 based on multi-source data fusion further includes: a monitoring result judgment module; The monitoring result judgment module is used to judge whether the vehicle has violated the traffic rules or engaged in dangerous behavior based on the vehicle monitoring results.

[0107] See also Figure 3 , Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 3The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of each module / unit in the above-mentioned system embodiments, such as Figure 2 The functions of modules 21 to 24 are shown.

[0108] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0109] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0110] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0111] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the vehicle monitoring method based on multi-source data fusion provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device described in the embodiments of the present disclosure, which will not be repeated here.

[0112] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0113] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0114] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0118] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A vehicle monitoring method based on multi-source data fusion, characterized in that: include: Acquire multiple types of data, where the multiple types of data come from different sources; Determine target data corresponding to the multiple types of data based on the multiple types of data; fusing target data corresponding to the multiple types of data to obtain comprehensive data; The vehicle is monitored based on the comprehensive data to obtain a vehicle monitoring result.

2. The vehicle monitoring method based on multi-source data fusion according to claim 1, characterized in that: The multiple types of data include image data and numerical data; The determining of multiple types of target data based on the multiple types of data includes: Evaluate the image data based on an image quality evaluation method to obtain image data quality; Evaluate the numerical data based on the numerical data evaluation method to obtain the quality of the numerical data; Determining a target image preprocessing strategy based on the image data quality; determining a target numerical preprocessing strategy based on the numerical data quality; Processing the image data based on the target image preprocessing strategy to obtain first type of target data; The numerical data is processed based on the target numerical preprocessing strategy to obtain second-category target data.

3. The vehicle monitoring method based on multi-source data fusion according to claim 1, characterized in that: Also includes: If the target data is real-time vehicle data, the vehicle feature is used as the target feature; If the target data is real-time road data, the road condition feature is used as the target feature; If the target data is real-time environmental data, the meteorological characteristics are used as target characteristics; Determining a target fusion strategy based on target features of the multiple types of data; Based on the target fusion strategy, multiple types of target data are fused to obtain the comprehensive data.

4. The vehicle monitoring method based on multi-source data fusion as claimed in claim 3, characterized in that: The target fusion strategy includes vehicle fusion strategy, road condition fusion strategy and weather fusion strategy; The determining of the target fusion strategy based on the target features of the multiple types of data includes: In response to the target feature being the vehicle feature, selecting the vehicle fusion strategy for fusion; In response to the target feature being the road condition feature, selecting the road condition fusion strategy for fusion; In response to the target feature being the meteorological feature, the meteorological fusion strategy is selected for fusion.

5. The vehicle monitoring method based on multi-source data fusion as claimed in claim 4, characterized in that: The method of fusing multiple types of target data based on the target fusion strategy to obtain the comprehensive data includes: Determining a vehicle fusion result of the vehicle fusion strategy; Determining a traffic condition fusion result of the traffic condition fusion strategy; Determining a meteorological fusion result of the meteorological fusion strategy; The vehicle fusion result, the road condition fusion result and the weather fusion result are weightedly calculated to obtain the comprehensive data.

6. The vehicle monitoring method based on multi-source data fusion according to claim 1, characterized in that: The monitoring of the vehicle based on the comprehensive data to obtain the vehicle monitoring result includes: The comprehensive data is input into a vehicle monitoring model to determine the vehicle monitoring result, wherein the vehicle monitoring model is trained based on historical vehicle comprehensive data.

7. The vehicle monitoring method based on multi-source data fusion according to claim 1, characterized in that: Also includes: Based on the vehicle monitoring results, it is determined whether the vehicle has committed any illegal or dangerous behavior.

8. A vehicle monitoring system based on multi-source data fusion, characterized in that: include: A data acquisition module, used to acquire multiple types of data, where the multiple types of data have different information sources; A target data determination module, used to determine target data corresponding to the multiple types of data based on the multiple types of data; A fusion module, used for fusing target data corresponding to the multiple types of data to obtain comprehensive data; The monitoring module is used to monitor the vehicle based on the comprehensive data to obtain vehicle monitoring results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images