A method and system for monitoring traffic flow at unmanned toll booths based on data fusion

Abstract

The present invention discloses a traffic flow monitoring method and system for unmanned toll booths based on data fusion, which relates to the field of intelligent transportation, including: deploying environmental sensors at key positions of unmanned toll booths, collecting environmental data in real time, using machine learning classification algorithms to classify environmental states, and generating environmental state labels; based on environmental state labels, adjusting the weight vectors of dynamic multimodal sensors through a multi-layer perceptron model, and generating a dynamic multimodal sensor weight configuration table; based on alarm information, combined with the dynamic data of all normal vehicles, dividing the number of normal vehicles passing through a time period into intervals, counting the flow density in each sub-interval, and generating a traffic flow monitoring report. The present invention realizes efficient data fusion under complex environmental adaptation by collecting environmental data and generating environmental state labels, combined with the generation and optimization of the dynamic multimodal sensor weight configuration table.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a method and system for monitoring traffic flow at an unmanned toll station based on data fusion. Background Art

[0002] With the rapid development of intelligent transportation systems, traffic flow monitoring technology has become an important means to achieve traffic management and optimization. Traditional toll station traffic monitoring mainly relies on a single sensor (such as video surveillance, lane counters, etc.) or manual recording. Although these methods are feasible in simple environments, in complex environments, especially in unmanned toll station scenarios, due to the changeable environmental conditions (such as rainy days, foggy days, low light at night, etc.), the data collection capability of a single sensor is often limited, and it is difficult to provide high-precision traffic flow monitoring results. In recent years, the application of data fusion technology has gradually attracted attention. By combining multimodal sensors (such as video, radar, infrared, etc.) to achieve multi-source complementarity and optimal utilization of data, the monitoring accuracy can be improved to a certain extent. However, most of the existing data fusion technologies use fixed weights or simple weighting methods, and fail to dynamically adjust the priority of sensors according to changes in the real-time environment, resulting in unsatisfactory monitoring effects under complex environmental conditions. At the same time, the existing technology lacks special optimization for unmanned toll collection scenarios in terms of abnormal behavior detection, and cannot effectively respond to the detection and response of abnormal behaviors such as vehicle reversal, congestion, and evasion, thereby affecting the efficient operation of toll stations.

[0003] Although some machine learning-based traffic flow monitoring methods have been explored in the academic and engineering fields, they mostly focus on the analysis of single features or the construction of static models, and have not yet fully utilized the characteristics of time series to predict dynamic behavior. For example, some existing technologies only analyze vehicle status based on single-frame video images, while ignoring the continuity characteristics of vehicle behavior, resulting in insufficient prediction of abnormal behavior. In addition, current traffic flow monitoring systems usually only count global traffic, lack separate analysis and alarm mechanisms for abnormal vehicles (such as speeding or toll evasion vehicles), which limits their applicability in specific scenarios. Therefore, existing technologies still have obvious deficiencies in dynamic environment adaptability, abnormal behavior detection, and accurate traffic statistics based on data fusion. A more comprehensive and adaptable traffic flow monitoring method is urgently needed to solve the above problems. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a traffic flow monitoring method for unmanned toll stations based on data fusion to solve the problems of inaccurate sensor fusion and insufficient abnormal behavior detection in dynamic environments.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for monitoring traffic flow at an unmanned toll station based on data fusion, which includes deploying environmental sensors at key locations of the unmanned toll station, collecting environmental data in real time, using a machine learning classification algorithm to classify environmental states, and generating environmental state labels; based on the environmental state labels, adjusting the weight vectors of the dynamic multimodal sensors through a multi-layer perceptron model, and generating a dynamic multimodal sensor weight configuration table; based on the dynamic multimodal sensor weight configuration table, starting the multimodal sensors to collect multimodal data in real time according to the weight distribution, preprocessing the collected multimodal data, and extracting vehicle feature vectors; constructing time series features based on the vehicle feature vectors, extracting time series windows using a fixed-length sliding window method, predicting abnormal behavior measurements, and generating alarm information for vehicles with abnormal behavior; based on the alarm information, combining the dynamic data of all normal vehicles, dividing the number of normal vehicles passing through the time period into intervals, counting the flow density in each sub-interval, and generating a traffic flow monitoring report.

[0008] As a preferred solution of the unmanned toll station traffic flow monitoring method based on data fusion described in the present invention, the collected environmental data includes light intensity, temperature, humidity, rainfall, wind speed, visibility and atmospheric pressure.

[0009] As a preferred solution of the traffic flow monitoring method of unmanned toll station based on data fusion described in the present invention, wherein: the environmental state is classified by using a machine learning classification algorithm to generate an environmental state label, and the specific steps are as follows:

[0010] According to the collected environmental data, the environmental status index is calculated using the machine learning classification algorithm. The expression is:

[0011] ;

[0012] in, represents the environmental status index, Indicates The weight of the feature, Indicates Related characteristic index, It means to emphasize the superposition effect of feature weight and feature index. Indicates normalization processing;

[0013] Based on historical environmental data, the original environmental classification threshold is defined as , ;

[0014] The ESV environmental status index is compared with the preset environmental classification threshold , Compare and generate environment status labels;

[0015] like When , the environment state is marked as low light at night;

[0016] like When , the environmental status is marked as rainy day with low visibility;

[0017] like When , the environmental state is marked as sunny with sufficient sunlight.

[0018] As a preferred solution of the traffic flow monitoring method of unmanned toll station based on data fusion described in the present invention, wherein: based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model to generate the dynamic multimodal sensor weight configuration table, the specific steps are as follows:

[0019] Based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model, and the expression is:

[0020] ;

[0021] in, Indicates The weight vector of the multimodal sensor dynamics, Indicates Multimodal sensor feature map matrix, Indicates Multimodal sensor feature vector, Indicates Multimodal sensor environment state mapping matrix, Represents the multi-classification label of the current environment, represents the bias vector, It indicates that the adjustable hyperparameter value of the control sensor feature is 0.8. Indicates that the adjustable hyperparameter value of the environment state is 0.5;

[0022] A dynamic multimodal sensor weight configuration table is generated according to the weight vector of the dynamic multimodal sensor.

[0023] As a preferred solution of the data fusion-based unmanned toll station traffic flow monitoring method of the present invention, wherein: based on the dynamic multimodal sensor weight configuration table, according to the weight distribution, the multimodal sensor is started to collect multimodal data in real time, the collected multimodal data is preprocessed, and the vehicle feature vector is extracted. The specific steps are as follows:

[0024] Based on the dynamic multimodal sensor weight configuration table, the multimodal sensor sampling frequency is adjusted. The expression is:

[0025] ;

[0026] in, Indicates Multimodal sensor sampling frequency, Indicates the maximum sampling frequency of the sensor, Indicates The weight values ​​corresponding to the multimodal sensors;

[0027] Based on the sampling frequency of multimodal sensors, all sensors collect data uniformly to form a data stream;

[0028] According to the data stream, the YOLOv8 deep learning model is used to detect the target of the video frame and extract the key features of the vehicle. The expression is:

[0029] ;

[0030] in, represents the video feature vector, represents the horizontal coordinate of the vehicle center, represents the vertical coordinate of the vehicle center, Indicates the width of the vehicle, Indicates the vehicle height, Indicates the vehicle category;

[0031] According to the data stream, the bimodal clustering algorithm is used to extract the vehicle speed and position features from the speed and distance data of the radar sensor. The expression is:

[0032] ;

[0033] in, represents the radar feature vector, represents the vehicle speed, Indicates the vehicle distance;

[0034] According to the data stream, the convolutional neural network is used to extract the vehicle contour features from the infrared thermal imaging data. The expression is:

[0035] ;

[0036] in, represents the infrared feature vector, represents the horizontal coordinate of the center of the vehicle contour, represents the vertical coordinate of the center of the vehicle contour, Indicates the calorific value intensity of the vehicle.

[0037] As a preferred solution of the traffic flow monitoring method of unmanned toll station based on data fusion described in the present invention, wherein: according to the vehicle feature vector, the time series feature is constructed, the fixed length sliding window method is adopted to extract the time series window, the abnormal behavior measurement is predicted, and the alarm information is generated for the vehicle with abnormal behavior. The specific steps are as follows:

[0038] Based on the extracted vehicle feature vector, the time series feature is constructed, and the expression is:

[0039] ;

[0040] in, Represents the time series characteristics, Indicates Video features of the moment, Indicates The radar signature of the moment, Indicates The infrared signature of the moment, represents the time series of vehicle behavior, Indicates the length of the time series;

[0041] Using the fixed-length sliding window method, the time series window is extracted from the continuous time series. The expression is:

[0042] ;

[0043] in, Indicates time series window, Indicates time point The set of vehicle multimodal feature vectors at time, Indicates time point The vehicle multimodal feature vector at time , Indicates time point The vehicle multimodal feature vector at time , Time window The starting time point, Indicates the length of the sliding window;

[0044] According to the time series window, the vehicle behavior feature vector at the next moment is predicted, and the expression is:

[0045] ;

[0046] in, represents the predicted feature vector for the next moment, represents a bidirectional long short-term memory network, Indicates time series windows;

[0047] According to the difference between the predicted vehicle behavior feature vector at the next moment and the actual observed value, the anomaly score is calculated, and the expression is:

[0048] ;

[0049] in, represents the abnormality score, Represents the dimension of the vehicle behavior feature vector, represents the actual eigenvector;

[0050] Based on the anomaly score With the preset anomaly score threshold Compare the data of the vehicle to determine whether the vehicle has abnormal behavior;

[0051] like When , the vehicle behaves normally;

[0052] like When , the vehicle behaves abnormally;

[0053] The real-time characteristic information of abnormal behavior vehicles, abnormal behavior types and timestamps are used to generate alarm information.

[0054] As a preferred solution of the traffic flow monitoring method of the unmanned toll station based on data fusion described in the present invention, wherein: based on the alarm information, combined with the dynamic data of all normal vehicles, the number of normal vehicles passing through the time period is divided into intervals, the flow density in each sub-interval is counted, and a traffic flow monitoring report is generated. The specific steps are as follows:

[0055] Based on the alarm information, abnormal vehicles are removed and the data of normal vehicles are retained to form a normal vehicle set. The expression is:

[0056] ;

[0057] in, Represents a dynamic data set of a normal vehicle, Indicates The time series feature set of a vehicle, A unique identifier for the vehicle. A set of identifiers representing abnormal vehicles;

[0058] Through the normal vehicle collection, the number of all normal vehicles passing in time period Y is counted;

[0059] Set the statistical time period Y to a fixed interval Divide into several sub-intervals , and count each subinterval The flow density within

[0060] The traffic density results of each time sub-interval are combined to form the traffic flow distribution of the toll station in the entire statistical time period, and a traffic flow monitoring report is generated.

[0061] In the second aspect, the present invention provides a traffic flow monitoring system for an unmanned toll station based on data fusion, including an environmental state label generation module, a dynamic weight configuration table generation module, a vehicle feature vector module, an alarm information generation module and a flow monitoring report generation module; the environmental state label generation module is used to deploy environmental sensors at key locations of unmanned toll stations, collect environmental data in real time, and use a machine learning classification algorithm to classify the environmental state and generate an environmental state label; the dynamic weight configuration table generation module is used to adjust the weight vector of the dynamic multimodal sensor through a multi-layer perceptron model based on the environmental state label to generate a dynamic multimodal sensor weight configuration table; the vehicle feature vector The measurement module is used to start the multimodal sensor to collect multimodal data in real time according to the weight distribution based on the dynamic multimodal sensor weight configuration table, pre-process the collected multimodal data, and extract the vehicle feature vector; the alarm information generation module is used to construct the time series characteristics according to the vehicle feature vector, use the fixed-length sliding window method to extract the time series window, predict the abnormal behavior measurement, and generate alarm information for vehicles with abnormal behavior; the flow monitoring report generation module is used to divide the number of normal vehicles passing through the time period into intervals based on the alarm information and combined with the dynamic data of all normal vehicles, count the flow density in each sub-interval, and generate a traffic flow monitoring report.

[0062] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the unmanned toll station traffic flow monitoring method based on data fusion as described in the first aspect of the present invention.

[0063] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the unmanned toll station traffic flow monitoring method based on data fusion as described in the first aspect of the present invention.

[0064] The beneficial effects of the present invention are as follows: by collecting environmental data and generating environmental status labels, combined with the generation and optimization of dynamic multimodal sensor weight configuration tables, efficient data fusion under complex environmental adaptation is achieved; time series modeling and abnormal behavior detection based on vehicle feature vectors further improve the accuracy and real-time performance of abnormal behavior identification. Finally, by eliminating abnormal data, combining all normal vehicle dynamic data, counting the traffic flow at the toll station and generating a monitoring report, an accurate and efficient traffic flow monitoring solution is provided for unmanned toll stations, significantly improving the traffic management capabilities under complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0066] Figure 1 This is a flow chart of the traffic flow monitoring method for unmanned toll stations based on data fusion in Example 1.

[0067] Figure 2 Schematic diagram of the unmanned toll station traffic flow monitoring system based on data fusion in Example 1. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0069] Example 1

[0070] Reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a method for monitoring traffic flow at an unmanned toll station based on data fusion, comprising the following steps:

[0071] S1. Deploy environmental sensors at key locations of unmanned toll booths to collect environmental data in real time, use machine learning classification algorithms to classify environmental states, and generate environmental state labels;

[0072] Collect environmental data including light intensity, temperature, humidity, rainfall, wind speed, visibility and atmospheric pressure;

[0073] According to the collected environmental data, the environmental status index is calculated using the machine learning classification algorithm. The expression is:

[0074] ;

[0075] in, represents the environmental status index, Indicates The weight of the feature, Indicates Related characteristic index, It means to emphasize the superposition effect of feature weight and feature index. Indicates normalization processing;

[0076] It should be explained that Used to quantify the complexity and condition differences of the current environment, Derived through historical data analysis, expert experience or machine learning model training, Directly reflects the actual value of a specific environmental feature under current conditions. For example, the lower the light intensity, the The closer it is to 0, It is used to compress the result of product operation to avoid the exponential superposition leading to excessive value. Represents the product operation.

[0077] Based on historical environmental data, the original environmental classification threshold is defined as , ;

[0078] It should be explained that Indicates the cutoff point of the first environmental classification, used to convert the environmental status index Divided into the turning point between "low state" and "medium state", Indicates the cut-off point of the second environmental classification, used to convert the environmental status index The turning point between "medium state" and "high state".

[0079] The ESV environmental status index is compared with the preset environmental classification threshold , Compare and generate environment status labels;

[0080] like When , the environment state is marked as low light at night;

[0081] like When , the environmental status is marked as rainy day with low visibility;

[0082] like When , the environmental state is marked as sunny with sufficient light;

[0083] It should be noted that the upper limit of low light at night This value should be set at a low enough level to ensure that Less than or equal to When it is really night or very dim environment. This value should be higher than , but still below the typical Values ​​are used to identify conditions where visibility is reduced due to rainfall or other factors.

[0084] S2. Based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model to generate a dynamic multimodal sensor weight configuration table;

[0085] Based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model, and the expression is:

[0086] ;

[0087] in, Indicates The weight vector of the multimodal sensor dynamics, Indicates Multimodal sensor feature map matrix, Indicates Multimodal sensor feature vector, Indicates Multimodal sensor environment state mapping matrix, Represents the multi-classification label of the current environment, represents the bias vector, It indicates that the adjustable hyperparameter value of the control sensor feature is 0.8. Indicates that the adjustable hyperparameter value of the environment state is 0.5;

[0088] It should be explained that The function represents the normalization of all calculation results to ensure the output weight vector The sum of the components of is 1, and each component lies in the interval [0, 1], which can be naturally interpreted as a probability distribution or a measure of relative importance. The function is a non-linear activation function that compresses the input range and introduces non-linear characteristics, which helps capture complex patterns and relationships.

[0089] Generate a dynamic multimodal sensor weight configuration table according to the weight vector of the dynamic multimodal sensor;

[0090] It should be noted that this method can flexibly adjust the importance of each sensor according to the actual environmental conditions to improve the overall performance. For example, in the "low light at night" condition, the weight of the infrared camera or thermal imager may be increased; while in the "sunny day with sufficient light", more reliance is placed on the visible light camera.

[0091] S3, based on the dynamic multimodal sensor weight configuration table, according to the weight distribution, start the multimodal sensor to collect multimodal data in real time, pre-process the collected multimodal data, and extract the vehicle feature vector;

[0092] Based on the dynamic multimodal sensor weight configuration table, the multimodal sensor sampling frequency is adjusted. The expression is:

[0093] ;

[0094] in, Indicates Multimodal sensor sampling frequency, Indicates the maximum sampling frequency of the sensor, Indicates The weight values ​​corresponding to the multimodal sensors;

[0095] It should be noted that the weight Reflecting the current environment The importance or reliability of the sensor data. A higher weight means that the information provided by the sensor is more critical and should be collected more frequently; conversely, a lower weight means that its sampling frequency can be appropriately reduced. The sampling frequency of each sensor is by setting its maximum sampling frequency Multiply by the corresponding weight This means that if the weight of a sensor is close to 1, its actual sampling frequency is almost equal to the maximum value; if the weight is smaller, the actual sampling frequency will decrease accordingly.

[0096] Based on the sampling frequency of multimodal sensors, all sensors collect data uniformly to form a data stream;

[0097] It should be noted that since all sensors work under the same clock control, the generated data stream has good time consistency, which is convenient for subsequent feature extraction, pattern recognition and other advanced processing; each data point has a clear timestamp, which is convenient for tracing its source and the specific time of occurrence, which is very useful for troubleshooting and performance evaluation; the close coordination of different types of sensor data in the time dimension can reveal phenomena or patterns that a single sensor cannot capture, thereby enhancing perception capabilities.

[0098] According to the data stream, the YOLOv8 deep learning model is used to detect the target of the video frame and extract the key features of the vehicle. The expression is:

[0099] ;

[0100] in, represents the video feature vector, represents the horizontal coordinate of the vehicle center, represents the vertical coordinate of the vehicle center, Indicates the width of the vehicle, Indicates the vehicle height, Indicates the vehicle category;

[0101] It should be explained that Contains information about the detected vehicle's location and category, is the position of the center point of the vehicle bounding box in the width direction of the image, The position of the center point of the vehicle bounding box in the image height direction, Used to distinguish different types of vehicles (such as cars, trucks, motorcycles, etc.), usually an integer or string label.

[0102] According to the data stream, the bimodal clustering algorithm is used to extract the vehicle speed and position features from the speed and distance data of the radar sensor. The expression is:

[0103] ;

[0104] in, represents the radar feature vector, represents the vehicle speed, Indicates the vehicle distance;

[0105] It should be noted that the bimodal clustering algorithm can effectively separate meaningful vehicle information from the chaotic data, reducing the possibility of misjudgment. Contains information about the speed and distance of detected vehicles, The speed at which the vehicle is moving relative to the radar sensor, The straight-line distance between the vehicle and the radar sensor.

[0106] According to the data stream, the convolutional neural network is used to extract the vehicle contour features from the infrared thermal imaging data. The expression is:

[0107] ;

[0108] in, represents the infrared feature vector, represents the horizontal coordinate of the center of the vehicle contour, represents the vertical coordinate of the center of the vehicle contour, Indicates the calorific value intensity of the vehicle;

[0109] It should be noted that a pre-trained convolutional neural network is used to collect a large number of labeled infrared thermal images as training samples, including different types of vehicles and their corresponding contour labels. These samples are used to perform supervised learning on the CNN so that it can learn to automatically recognize and segment the vehicle contours in the image.

[0110] S4. Based on the vehicle feature vector, construct the time series feature, use the fixed-length sliding window method to extract the time series window, predict the abnormal behavior measurement, and generate alarm information for the vehicle with abnormal behavior;

[0111] Based on the extracted vehicle feature vector, the time series feature is constructed, and the expression is:

[0112] ;

[0113] in, Represents the time series characteristics, Indicates Video features of the moment, Indicates The radar signature of the moment, Indicates The infrared signature of the moment, represents the time series of vehicle behavior, Indicates the length of the time series;

[0114] It should be explained that Represents a series of feature vectors about a specific vehicle collected by multiple sensors over a period of time. Usually includes information such as position, shape, color, etc. extracted from images obtained by visible light cameras. May contain distance, speed, acceleration and other motion-related information. As discussed previously, the coordinates of the center of the vehicle contour and its thermal value intensity are represented.

[0115] Using the fixed-length sliding window method, the time series window is extracted from the continuous time series. The expression is:

[0116] ;

[0117] in, Indicates time series window, Indicates time point The set of vehicle multimodal feature vectors at time, Indicates time point The vehicle multimodal feature vector at time , Indicates time point The vehicle multimodal feature vector at time , Time window The starting time point, Indicates the length of the sliding window;

[0118] It should be noted that by dividing the time series into multiple windows of fixed length, the behavior patterns of vehicles that change over time, such as acceleration, deceleration, turning, etc., can be better captured. When new data arrives, only the most recent window needs to be updated and the earliest window removed, so that a rolling time window can be maintained, which is suitable for real-time monitoring systems. The window length can be adjusted To adapt to different application scenarios and analysis objectives, for example, short-term trend analysis can choose a shorter window, while long-term trend analysis requires a longer window.

[0119] According to the time series window, the vehicle behavior feature vector at the next moment is predicted, and the expression is:

[0120] ;

[0121] in, represents the predicted feature vector for the next moment, represents a bidirectional long short-term memory network, Indicates time series windows;

[0122] It should be explained that It stands for Bidirectional Long Short-Term Memory Network, which is a special kind of recurrent neural network that can consider both past and future contextual information when processing sequence data. Represents the predicted next moment feature vector, indicating the time point The multimodal characteristics of the vehicle such as position, speed, and calorific value intensity are taken into consideration.

[0123] According to the difference between the predicted vehicle behavior feature vector at the next moment and the actual observed value, the anomaly score is calculated, and the expression is:

[0124] ;

[0125] in, represents the abnormality score, Represents the dimension of the vehicle behavior feature vector, represents the actual eigenvector;

[0126] It should be explained that The mean absolute error of the difference between the predicted values ​​and the actual observed values ​​is used to measure the degree of anomaly; That is, how many independent features (such as position, speed, heat value intensity, etc.) each feature vector contains; At the point in time Time The actual feature vector of dimension.

[0127] Based on the anomaly score With the preset anomaly score threshold Compare the data of the vehicle to determine whether the vehicle has abnormal behavior;

[0128] like When , the vehicle behaves normally;

[0129] like When , the vehicle behaves abnormally;

[0130] It should be noted that one or more abnormal score thresholds should be reasonably set according to the needs of specific application scenarios and historical data statistical analysis. This threshold should be able to effectively distinguish normal from abnormal behavior while avoiding too many false positives or false negatives.

[0131] Generate alarm information based on the real-time characteristic information, abnormal behavior type and timestamp of abnormal behavior vehicles;

[0132] It should be noted that when abnormal behavior is detected, relevant information of the vehicle is collected, including but not limited to real-time feature information, such as multimodal sensor data such as location, speed, thermal value intensity, etc.; the type of abnormal behavior, based on the specific dimensional differences in the abnormality score, identifies the type of abnormality (such as sudden braking, sharp turns, lane deviation, etc.); timestamp, records the exact time when the abnormality occurred, to facilitate subsequent tracking and analysis.

[0133] S5. Based on the alarm information and combined with the dynamic data of all normal vehicles, the number of all normal vehicles passing through the time period is divided into intervals, the flow density in each sub-interval is counted, and a traffic flow monitoring report is generated;

[0134] Based on the alarm information, abnormal vehicles are removed and the data of normal vehicles are retained to form a normal vehicle set. The expression is:

[0135] ;

[0136] in, Represents a dynamic data set of a normal vehicle, Indicates The time series feature set of a vehicle, A unique identifier for the vehicle. A set of identifiers representing abnormal vehicles;

[0137] It should be explained that Contains the time series features of all vehicles that are judged to have normal behavior, Including the multimodal feature vectors of the vehicle at each time point (such as position, speed, thermal value intensity, etc.), Contains the unique identifiers of all vehicles that have been judged to have abnormal behavior.

[0138] Through the normal vehicle collection, the number of all normal vehicles passing in time period Y is counted;

[0139] It should be stated that the time period Y to be counted should be clear, such as 8 am to 6 pm on a certain day, or weekdays in a week. For each normal vehicle, extract the first appearing timestamp from its time series feature set. For each timestamp, check whether it falls within the specified time period Y. If so, count the vehicle in the traffic count. If not, ignore the vehicle. Add up the number of vehicles that meet the conditions to get the final traffic count.

[0140] Set the statistical time period Y to a fixed interval Divide into several sub-intervals , and count each subinterval The flow density within

[0141] It should be noted that by subdividing the sub-intervals, the peak and trough periods of traffic flow can be identified more accurately, which is of great significance for optimizing the setting of traffic lights, adjusting public transportation schedules, etc. Capturing changes in traffic flow in a short period of time, such as morning peak, lunch break, evening peak, etc., helps traffic.

[0142] Combine the traffic density results of each time sub-interval to form the traffic flow distribution of the toll station in the entire statistical time period, and generate a traffic flow monitoring report;

[0143] It should be noted that a detailed traffic flow monitoring report is automatically generated every day to help traffic management departments to grasp road conditions in a timely manner and make reasonable dispatching decisions. Before major holidays, the possible traffic pressure is predicted based on historical traffic density data, and police forces and diversion measures are deployed in advance to ensure smooth roads. For newly built roads, the actual use of them is evaluated through traffic density statistics over a period of time to provide data support for subsequent improvements.

[0144] The present embodiment also provides a traffic flow monitoring system for an unmanned toll station based on data fusion, including: an environmental state label generation module, a dynamic weight configuration table generation module, a vehicle feature vector module, an alarm information generation module and a flow monitoring report generation module; the environmental state label generation module is used to deploy environmental sensors at key locations of unmanned toll stations, collect environmental data in real time, classify environmental states using machine learning classification algorithms, and generate environmental state labels; the dynamic weight configuration table generation module is used to adjust the weight vectors of dynamic multimodal sensors based on environmental state labels through a multi-layer perceptron model to generate a dynamic multimodal sensor weight configuration table; the vehicle feature vector module is used to generate a dynamic multimodal sensor weight configuration table based on environmental state labels; The block is used to start the multimodal sensor to collect multimodal data in real time according to the weight distribution based on the dynamic multimodal sensor weight configuration table, pre-process the collected multimodal data, and extract the vehicle feature vector; the alarm information generation module is used to construct the time series feature according to the vehicle feature vector, use the fixed-length sliding window method to extract the time series window, predict the abnormal behavior measurement, and generate alarm information for the vehicle with abnormal behavior; the flow monitoring report generation module is used to divide the number of normal vehicles passing through the time period into intervals based on the alarm information and the dynamic data of all normal vehicles, count the flow density in each sub-interval, and generate a traffic flow monitoring report.

[0145] This embodiment also provides a computer device, which is suitable for the case of an unmanned toll station traffic flow monitoring method based on data fusion, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the unmanned toll station traffic flow monitoring method based on data fusion as proposed in the above embodiment.

[0146] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0147] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for monitoring traffic flow at an unmanned toll station based on data fusion proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage device, a flash memory, a disk or an optical disk.

[0148] In summary, the present invention achieves efficient data fusion under complex environment adaptation by: collecting environmental data and generating environmental status labels, combining the generation and optimization of dynamic multimodal sensor weight configuration tables; and further improves the accuracy and real-time performance of abnormal behavior identification based on time series modeling and abnormal behavior detection based on vehicle feature vectors. Finally, by eliminating abnormal data, combining all normal vehicle dynamic data, counting the traffic flow at the toll station and generating a monitoring report, it provides an accurate and efficient traffic flow monitoring solution for unmanned toll stations, significantly improving the traffic management capabilities in complex environments.

[0149] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A traffic flow monitoring method for unmanned toll booths based on data fusion, characterized in that: include: Deploy environmental sensors at key locations of unmanned toll booths to collect environmental data in real time, use machine learning classification algorithms to classify environmental states, and generate environmental state labels; Based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model to generate a dynamic multimodal sensor weight configuration table; Based on the dynamic multimodal sensor weight configuration table, according to the weight distribution, the multimodal sensor is started to collect multimodal data in real time, the collected multimodal data is preprocessed, and the vehicle feature vector is extracted; Based on the vehicle feature vector, the time series features are constructed, and the fixed-length sliding window method is used to extract the time series window, predict the abnormal behavior measurement, and generate alarm information for vehicles with abnormal behavior; Based on the alarm information and combined with the dynamic data of all normal vehicles, the number of all normal vehicles passing through the time period is divided into intervals, the flow density in each sub-interval is counted, and a traffic flow monitoring report is generated; The collected environmental data include light intensity, temperature, humidity, rainfall, wind speed, visibility and atmospheric pressure; The machine learning classification algorithm is used to classify the environmental state and generate the environmental state label. The specific steps are as follows: According to the collected environmental data, the environmental status index is calculated using the machine learning classification algorithm. The expression is: ； in, represents the environmental status index, Indicates The weight of the feature, Indicates Related characteristic index, It means to emphasize the superposition effect of feature weight and feature index. Indicates normalization processing; Based on historical environmental data, the original environmental classification threshold is defined as , ; The ESV environmental status index is compared with the preset environmental classification threshold , Compare and generate environment status labels; like When , the environment state is marked as low light at night; like When , the environmental status is marked as rainy day with low visibility; like When , the environmental state is marked as sunny with sufficient light; Based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model to generate a dynamic multimodal sensor weight configuration table. The specific steps are as follows: Based on the environmental state label, the weight vector of the dynamic multimodal sensor is adjusted through the multi-layer perceptron model, and the expression is: ； in, Indicates The weight vector of the multimodal sensor dynamics, Indicates Multimodal sensor feature map matrix, Indicates Multimodal sensor feature vector, Indicates Multimodal sensor environment state mapping matrix, Represents the multi-classification label of the current environment, represents the bias vector, It indicates that the adjustable hyperparameter value of the control sensor feature is 0.

8. Indicates that the adjustable hyperparameter value of the environment state is 0.5; A dynamic multimodal sensor weight configuration table is generated according to the weight vector of the dynamic multimodal sensor.

2. The method for monitoring traffic flow at unmanned toll booths based on data fusion as claimed in claim 1, characterized in that: Based on the dynamic multimodal sensor weight configuration table, according to the weight distribution, the multimodal sensor is started to collect multimodal data in real time, the collected multimodal data is preprocessed, and the vehicle feature vector is extracted. The specific steps are as follows: Based on the dynamic multimodal sensor weight configuration table, the multimodal sensor sampling frequency is adjusted. The expression is: ； in, Indicates Multimodal sensor sampling frequency, Indicates the maximum sampling frequency of the sensor, Indicates The weight values ​​corresponding to the multimodal sensors; Based on the sampling frequency of multimodal sensors, all sensors collect data uniformly to form a data stream; According to the data stream, the YOLOv8 deep learning model is used to detect the target of the video frame and extract the key features of the vehicle. The expression is: ； in, represents the video feature vector, represents the horizontal coordinate of the vehicle center, represents the vertical coordinate of the vehicle center, Indicates the width of the vehicle, Indicates the vehicle height, Indicates the vehicle category; According to the data stream, the bimodal clustering algorithm is used to extract the vehicle speed and position features from the speed and distance data of the radar sensor. The expression is: ； in, represents the radar feature vector, represents the vehicle speed, Indicates the vehicle distance; According to the data stream, the convolutional neural network is used to extract the vehicle contour features from the infrared thermal imaging data. The expression is: ； in, represents the infrared feature vector, represents the horizontal coordinate of the center of the vehicle contour, represents the vertical coordinate of the center of the vehicle contour, Indicates the vehicle's calorific value intensity.

3. The method for monitoring traffic flow at unmanned toll booths based on data fusion as claimed in claim 2, characterized in that: The method constructs time series features based on vehicle feature vectors, uses a fixed-length sliding window method to extract time series windows, predicts abnormal behavior measurements, and generates alarm information for vehicles with abnormal behavior. The specific steps are as follows: Based on the extracted vehicle feature vector, the time series feature is constructed, and the expression is: ； in, Represents the time series characteristics, Indicates Video features of the moment, Indicates The radar signature of the moment, Indicates The infrared signature of the moment, represents the time series of vehicle behavior, Indicates the length of the time series; Using the fixed-length sliding window method, the time series window is extracted from the continuous time series. The expression is: ； in, Indicates time series window, Indicates time point The set of vehicle multimodal feature vectors at time, Indicates time point The vehicle multimodal feature vector at time , Indicates time point The vehicle multimodal feature vector at time , Time Window The starting time point, Indicates the length of the sliding window; According to the time series window, the vehicle behavior feature vector at the next moment is predicted, and the expression is: ； in, represents the predicted feature vector for the next moment, represents a bidirectional long short-term memory network, Indicates time series windows; According to the difference between the predicted vehicle behavior feature vector at the next moment and the actual observed value, the anomaly score is calculated, and the expression is: ； in, represents the abnormality score, Represents the dimension of the vehicle behavior feature vector, represents the actual eigenvector; Based on the anomaly score With the preset anomaly score threshold Compare the data of the vehicle to determine whether the vehicle has abnormal behavior; like When , the vehicle behaves normally; like When , the vehicle behaves abnormally; The real-time characteristic information of abnormal behavior vehicles, abnormal behavior types and timestamps are used to generate alarm information.

4. The method for monitoring traffic flow at unmanned toll booths based on data fusion as claimed in claim 3, characterized in that: Based on the alarm information, combined with the dynamic data of all normal vehicles, the number of all normal vehicles passing in the time period is divided into intervals, the flow density in each sub-interval is counted, and a traffic flow monitoring report is generated. The specific steps are as follows: Based on the alarm information, abnormal vehicles are removed and the data of normal vehicles are retained to form a normal vehicle set. The expression is: ； in, Represents a dynamic data set of a normal vehicle, Indicates The time series feature set of a vehicle, A unique identifier for the vehicle. A set of identifiers representing abnormal vehicles; Through the normal vehicle collection, the number of all normal vehicles passing in time period Y is counted; Set the statistical time period Y to a fixed interval Divide into several sub-intervals , and count each subinterval The flow density within The traffic density results of each time sub-interval are combined to form the traffic flow distribution of the toll station in the entire statistical time period, and a traffic flow monitoring report is generated.

5. A data fusion-based unmanned toll station traffic flow monitoring system, based on the data fusion-based unmanned toll station traffic flow monitoring method according to any one of claims 1 to 4, characterized in that: Including, environment status label generation module, dynamic weight configuration table generation module, vehicle feature vector module, alarm information generation module and traffic monitoring report generation module; The environment status label generation module is used to deploy environmental sensors at key locations of unmanned toll booths, collect environmental data in real time, and use machine learning classification algorithms to classify environmental status and generate environmental status labels; A dynamic weight configuration table generation module is used to adjust the weight vector of the dynamic multimodal sensor through a multi-layer perceptron model based on the environmental state label to generate a dynamic multimodal sensor weight configuration table; The vehicle feature vector module is used to start the multimodal sensor to collect multimodal data in real time according to the weight distribution based on the dynamic multimodal sensor weight configuration table, pre-process the collected multimodal data, and extract the vehicle feature vector; The alarm information generation module is used to construct time series features based on vehicle feature vectors, extract time series windows using a fixed-length sliding window method, predict abnormal behavior measurements, and generate alarm information for vehicles with abnormal behavior; The traffic monitoring report generation module is used to divide the number of normal vehicles passing through a time period into intervals based on alarm information and combined with the dynamic data of all normal vehicles, count the flow density in each sub-interval, and generate a traffic flow monitoring report.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the unmanned toll station traffic flow monitoring method based on data fusion as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the unmanned toll station traffic flow monitoring method based on data fusion as described in any one of claims 1 to 4 are implemented.

Images