Vehicle overspeed management method and system based on multi-modal fusion

Abstract

The invention relates to the field of vehicle management and control, and discloses a vehicle overspeed management method and system based on multi-modal fusion, and the method comprises the following steps: monitoring the traffic flow of a main road of a construction site, recording overspeed vehicles, and carrying out the overspeed management of the vehicles in a vehicle management system of the construction site in combination with a multi-modal data fusion analysis method. Compared with a single-modal identification technology, the multi-modal fusion technology can make full use of complementarity and redundancy among different modal data, and improves the precision and comprehensiveness of information expression. Meanwhile, the technology can also adapt to different environments and scenes, and has higher adaptability and flexibility.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control, and in particular to a vehicle speeding management method and system based on multimodal fusion. Background Art

[0002] Large-scale construction cluster projects are characterized by large construction sites, many temporary roads, concentrated on-site vehicles, and great management difficulties. Especially in multiple sub-construction areas on site, there are a large number of vehicles in each area, and there are many vehicle intersections on the main roads and branch roads of the construction site. Therefore, the problem of vehicle speeding on site is relatively serious. Due to problems such as dust on the construction site, the license plates of the incoming vehicles will be blocked to a certain extent. How to manage the phenomenon of vehicle speeding is an urgent problem to be solved in large-area construction sites. This solution mainly involves a speeding radar, a high-speed camera, and a construction site vehicle management system. Among them, the speeding radar can calculate the frequency change of the reflected signal and can monitor the speed of vehicles on the road in real time; when the radar detects that the vehicle speed exceeds the set speed limit value, the high-speed camera will immediately trigger the speeding determination mechanism, and the high-speed camera will start to take pictures of the speeding vehicle; the construction site vehicle management system mainly records the basic vehicle information such as the color, model, license plate, and engine soundprint of the incoming vehicles, and manages vehicle speeding based on the multimodal fusion method. Therefore, a vehicle speeding management method and system based on multimodal fusion are proposed. Summary of the Invention

[0003] The present invention overcomes the deficiencies of the prior art and provides a vehicle speeding management method and system based on multimodal fusion.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows: In the first aspect of the present invention, a vehicle speeding management method based on multimodal fusion is provided, including the following steps: S102: Conduct real-time monitoring of the traffic flow on the main roads of the construction site, and determine the installation positions of the speeding radar and the high-speed camera in combination with the real-time monitoring results of the traffic flow; S104: Conduct information interconnection optimization tests on the target speeding radar and the target high-speed camera to realize the construction of a qualified construction site vehicle management system; S106: Run the qualified construction site vehicle management system, and record and display the vehicle dynamics of speeding vehicles in the qualified construction site vehicle management system in combination with the multimodal data fusion analysis method.

[0005] Further, in a preferred embodiment of the present invention, the S102 is specifically: Determine the main road at the construction site and introduce the historical data network. Based on the historical data network, determine all the positions on the main road at the construction site where speed radars and high-speed cameras can be installed, and label them as speed radar installable positions and high-speed camera installable positions; Determine the working ranges of different speed radar installable positions and high-speed camera installable positions on the main road at the construction site, and retrieve the historical traffic flow in the corresponding working ranges in the historical data network; Preset a target traffic flow, and divide the speed radar installable positions and high-speed camera installable positions corresponding to the historical traffic flow not less than the target traffic flow in the working ranges on the main road at the construction site into target speed radar installation positions and target high-speed camera installation positions; Install speed radars and high-speed cameras at the target speed radar installation positions and target high-speed camera installation positions to obtain target speed radars and target high-speed cameras.

[0006] Further, in a preferred embodiment of the present invention, the S104 is specifically: Use the same communication protocol for the target speed radar and the target high-speed camera, and label it as the target communication protocol, wherein the target communication protocol ensures the same-frequency information interconnection between the target speed radar and the target high-speed camera; Conduct a test operation on the target speed radar and the target high-speed camera, and synchronize the data collection times of the target speed radar and the target high-speed camera during the test operation in combination with the target communication protocol; When the data collection times of the target speed radar and the target high-speed camera are synchronized during the test operation, obtain the management system of the main road at the construction site, and perform data interconnection between the target speed radar and the target high-speed camera and the management system of the main road at the construction site to construct a vehicle management system for the construction site; In the vehicle management system for the construction site, control the target speed radar to output a speed radar test data packet during the test operation, wherein the speed radar test data packet contains the real-time test speed value of the vehicle at the construction site and the corresponding timestamp, and after testing and outputting the speed radar test data packet, the response time for the target high-speed camera to generate a capture instruction is labeled as the test response time; Preset a standard test response time threshold. If the test response time remains within the standard test response time threshold, it is determined that the information interconnection effect of the target speed radar and the target high-speed camera is qualified, and the vehicle management system for the construction site is divided into a qualified vehicle management system for the construction site; If the test response time does not remain within the standard test response time threshold, then introduce time-sensitive network technology into the construction site vehicle management system to improve the time synchronization accuracy of the target speed radar and the target high-speed camera until the test response time remains within the standard test response time threshold.

[0007] Further, in a preferred embodiment of the present invention, the S106 is specifically as follows: Run a qualified construction site vehicle management system, based on the target speed radar, emit and receive reflected electromagnetic waves on the main road of the construction site in real time, and at the same time calculate the frequency of the reflected electromagnetic waves in the qualified construction site vehicle management system to generate the real-time speed values of different vehicles on the main road of the construction site; Preset a speed limit threshold in the qualified construction site vehicle management system. If there is no real-time speed value of a vehicle greater than the speed limit threshold, then mark all vehicles passing through the target speed radar as non-over speeding vehicles in the qualified construction site vehicle management system and perform storage processing at the same time; When there is a real-time speed value of a vehicle greater than the speed limit threshold, mark the corresponding vehicle as an over speeding vehicle, and at the same time start the target high-speed camera in the qualified construction site vehicle management system, and control the target high-speed camera to capture an image of the over speeding vehicle in real time, which is calibrated as an over speeding vehicle image; Perform image recognition, image feature extraction, and image feature classification on the over speeding vehicle image to obtain the vehicle information of the over speeding vehicle image; Construct an over speeding vehicle trajectory prediction model for predicting the driving trajectory of the over speeding vehicle, and combine the extracted vehicle information of the over speeding vehicle and the driving trajectory of the over speeding vehicle to record the vehicle dynamics of the over speeding vehicle in the qualified construction site vehicle management system.

[0008] Further, in a preferred embodiment of the present invention, the performing image recognition, image feature extraction, and image feature classification on the over speeding vehicle image to obtain the vehicle information of the over speeding vehicle image is specifically as follows: In the qualified construction site vehicle management system, introduce a motion blur elimination algorithm to eliminate the motion blur of the over speeding vehicle image, and perform wavelet denoising processing on the over speeding vehicle image after motion blur elimination to obtain a denoised over speeding vehicle image; Extract the RGB values of all pixel points in the denoised over speeding vehicle image, calculate the proportion of different RGB values, obtain a color dictionary in the big data network, and import the proportion of different RGB values into the color dictionary for retrieval and comparison to obtain the main color of the over speeding vehicle; Extract the pixel points at the headlights, intake grilles, and vehicle logos in the images of noise-reduced speeding vehicles, and introduce the SIFT feature matching algorithm. Retrieve the pixel points at the headlights, intake grilles, and vehicle logos of all brand models of vehicles in the big data network, and perform feature matching based on the SIFT feature matching algorithm with the pixel points at the headlights, intake grilles, and vehicle logos in the images of noise-reduced speeding vehicles to generate the matching rates between the speeding vehicles and all brand models of vehicles; Analyze the matching rates between the speeding vehicles and all brand models of vehicles, and calibrate the brand model of the vehicle with the highest matching rate as the brand model of the speeding vehicle; Introduce the Sobel operator to extract the image features of the license plate position of the speeding vehicle, and perform character analysis on the image features of the license plate position of the speeding vehicle to determine the license plate number of the speeding vehicle; Classify the main color, brand model, and license plate number of the speeding vehicle to generate the vehicle information of the speeding vehicle.

[0009] Furthermore, in a preferred embodiment of the present invention, the construction of the speeding vehicle trajectory prediction model is used to predict the driving trajectory of the speeding vehicle. Combining the extracted vehicle information of the speeding vehicle and the driving trajectory of the speeding vehicle, record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system, specifically: Obtain the GPS longitude and latitude of the target highway camera, record the real-time speed value of the speeding vehicle, and obtain all the roads connecting the main roads of the construction site, which are designated as the roads to be analyzed; Encode the connection relationships of all the roads to be analyzed and the main roads of the construction site through a graph neural network to generate a road topology map, and obtain the real-time traffic light states of all the roads to be analyzed and the main roads of the construction site; Introduce a multi-modal spatio-temporal fusion blank model, and convert the road topology map, the GPS longitude and latitude of the target highway camera, the real-time speed value of the speeding vehicle, and the real-time traffic light states of all the roads to be analyzed and the main roads of the construction site into feature data in a time series feature state, which is designated as the target time series feature data; In the multi-modal spatio-temporal fusion blank model, perform adaptive Kalman filtering on the target time series feature data to eliminate the feature data noise and obtain the target filtered time series feature data. At the same time, perform spatial interaction modeling on the target filtered time series feature data in the multi-modal spatio-temporal fusion blank model to obtain the speeding vehicle trajectory prediction model; Generate all modal trajectories of the speeding vehicle based on the speeding vehicle trajectory prediction model; Introduce the Gaussian mixture algorithm to calculate the distribution weights of the speeding vehicle in different modal trajectories, where the distribution weights of different modal trajectories represent the probability values of different driving trajectories of the speeding vehicle; Obtain the modal trajectory corresponding to the maximum distribution weight, and obtain the target driving trajectory of the speeding vehicle. Combine the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle to generate and record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0010] The second aspect of the present invention also provides a vehicle speeding management system based on multi-modal fusion. The vehicle speeding management system includes a memory and a processor. The vehicle speeding management method is stored in the memory. When the vehicle speeding management method is executed by the processor, the following steps are realized: Perform real-time monitoring of the traffic flow on the main road of the construction site, and determine the installation positions of the speeding radar and the high-speed camera in combination with the real-time traffic flow monitoring results; Conduct information interconnection optimization tests on the target speeding radar and the target high-speed camera to realize the construction of a qualified construction site vehicle management system; Run the qualified construction site vehicle management system, and record and display the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system in combination with the multi-modal data fusion analysis method.

[0011] The present invention solves the technical defects existing in the background art. The present invention has the following beneficial effects: monitor the traffic flow on the main road of the construction site, record the speeding vehicles, and combine the multi-modal data fusion analysis method in the construction site vehicle management system to manage the speeding of vehicles. Compared with the single-modal recognition technology, the multi-modal fusion technology can make full use of the complementarity and redundancy between different modal data, improve the accuracy and comprehensiveness of information expression. At the same time, this technology can also adapt to different environments and scenarios, and has stronger adaptability and flexibility. Brief Description of the Drawings

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0013] Figure 1 Shows the flowchart of the vehicle speeding management method based on multi-modal fusion; Figure 2 Shows the method flowchart of recording and displaying the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system in combination with the multi-modal data fusion analysis method; Figure 3 Shows the program view of the vehicle speeding management system based on multi-modal fusion. Detailed Description of the Embodiments

[0014] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0015] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0016] Figure 1 A flowchart showing a vehicle speeding management method based on multimodal fusion is shown, including the following steps: S102: Real-time monitor the traffic flow on the main road of the construction site, and determine the installation positions of the speeding radar and the high-speed camera in combination with the real-time traffic flow monitoring results; S104: Conduct information interconnection optimization tests on the target speeding radar and the target high-speed camera to realize the construction of a qualified vehicle management system for the construction site; S106: Run the qualified vehicle management system for the construction site, and record and display the vehicle dynamics of speeding vehicles in the qualified vehicle management system for the construction site in combination with the multimodal data fusion analysis method.

[0017] Further, in a preferred embodiment of the present invention, the S102 is specifically: Determine the main road of the construction site, introduce the historical data network, and based on the historical data network, determine all the positions on the main road of the construction site where the speeding radar and the high-speed camera can be installed, and mark them as the installable positions of the speeding radar and the installable positions of the high-speed camera; Determine the working ranges of different installable positions of the speeding radar and the installable positions of the high-speed camera on the main road of the construction site, and retrieve the historical traffic flow in the corresponding working ranges in the historical data network; Preset the target traffic flow, and divide the installable positions of the speeding radar and the installable positions of the high-speed camera corresponding to the historical traffic flow not less than the target traffic flow in the working ranges on the main road of the construction site into the target speeding radar installation positions and the target high-speed camera installation positions; Install the speeding radar and the high-speed camera at the corresponding target speeding radar installation positions and the target high-speed camera installation positions to obtain the target speeding radar and the target high-speed camera.

[0018] It should be noted that large-scale construction cluster projects are characterized by a large construction site, many temporary roads, concentrated on-site vehicles, and great management difficulties. Especially in multiple sub-construction areas on-site, with a large number of vehicles in each area, there are many vehicle intersections on the main roads and branch roads of the construction site. Therefore, the problem of vehicle speeding on-site is relatively serious. To avoid dangerous situations caused by vehicle speeding on-site, it is necessary to monitor whether there is speeding through a speeding radar and confirm vehicle information through a high-speed camera, so as to remind and punish speeding vehicles. First, it is necessary to determine the installation positions of the speeding radar and the high-speed camera to ensure that all vehicles can be recorded and photographed on the main roads of the construction site. The installation positions can be determined according to the traffic flow. After determining the installation positions, the speeding radar and the camera need to be installed and processed.

[0019] Further, in a preferred embodiment of the present invention, the S104 is specifically: Use the same communication protocol for the target speeding radar and the target high-speed camera, and calibrate it as the target communication protocol, where the target communication protocol ensures the same-frequency information interconnection of the target speeding radar and the target high-speed camera; Test and operate the target speeding radar and the target high-speed camera, and combine the target communication protocol to synchronize the data acquisition times of the target speeding radar and the target high-speed camera during the test operation; When the data acquisition times of the target speeding radar and the target high-speed camera are synchronized during the test operation, obtain the management system of the main road of the construction site, and perform data interconnection between the target speeding radar and the target high-speed camera and the management system of the main road of the construction site to construct a vehicle management system for the construction site; In the vehicle management system for the construction site, control the target speeding radar to output a speeding radar test data packet during the test operation, where the speeding radar test data packet contains the real-time test speed value and the corresponding timestamp of the vehicle on the construction site, and after testing and outputting the speeding radar test data packet, the response time of the target high-speed camera to generate a capture instruction is calibrated as the test response time; Preset a standard test response time threshold. If the test response time is maintained within the standard test response time threshold, it is determined that the information interconnection effect of the target speeding radar and the target high-speed camera is qualified, and the vehicle management system for the construction site is classified as a qualified vehicle management system for the construction site; If the test response time is not maintained within the standard test response time threshold, introduce time-sensitive network technology into the vehicle management system for the construction site to improve the time synchronization accuracy of the target speeding radar and the target high-speed camera until the test response time is maintained within the standard test response time threshold.

[0020] It should be noted that the target speed radar and the target high-speed camera achieve information interconnection with the same frequency. The purpose is to ensure that after the speed radar detects a vehicle speeding, the high-speed camera can take pictures immediately. Otherwise, if the vehicle passes by the high-speed camera, it will be impossible to clearly capture the speeding vehicle or even unable to capture it at all. Ensure that the data collection time of the target speed radar and the target high-speed camera is synchronized during the test operation, and construct a vehicle management system for the construction site to record the basic vehicle information such as the color, model, and license plate of the vehicles entering the site, and manage vehicle speeding based on the multimodal fusion method. The time-sensitive network technology is an algorithm that improves the time synchronization accuracy of the target speed radar and the target high-speed camera, and is used to improve the sensitivity of the time synchronization accuracy of the target speed radar and the target high-speed camera to data. The target communication protocol uses the PTP light communication protocol to achieve time synchronization at the microsecond level.

[0021] Figure 2 The flowchart of the method for recording and displaying the vehicle dynamics of speeding vehicles in a qualified construction site vehicle management system by combining the multimodal data fusion analysis method is shown, including the following steps: S202: Run the qualified construction site vehicle management system, and combine the multimodal data fusion analysis method to record and display the vehicle dynamics of speeding vehicles in the qualified construction site vehicle management system; S204: Perform image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain the vehicle information of the speeding vehicle image; S206: Construct a speeding vehicle trajectory prediction model for predicting the driving trajectory of the speeding vehicle, and combine the extracted vehicle information of the speeding vehicle and the driving trajectory of the speeding vehicle to record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0022] Furthermore, in a preferred embodiment of the present invention, the S106 is specifically: Run the qualified construction site vehicle management system, and based on the target speed radar, continuously emit and receive reflected electromagnetic waves on the main road of the construction site. At the same time, calculate the frequency of the reflected electromagnetic waves in the qualified construction site vehicle management system to generate the real-time speed values of different vehicles on the main road of the construction site; Preset a speed limit threshold in the qualified construction site vehicle management system. If there is no real-time speed value of a vehicle greater than the speed limit threshold, then in the qualified construction site vehicle management system, mark all vehicles passing by the target speed radar as non-speeding vehicles and perform storage processing at the same time; When there is a real-time speed value of a vehicle greater than the speed limit threshold, mark the corresponding vehicle as a speeding vehicle. At the same time, start the target high-speed camera in the qualified construction site vehicle management system, and control the target high-speed camera to take pictures of the speeding vehicle in real time, and calibrate it as a speeding vehicle image; Perform image recognition, image feature extraction, and image feature classification on the images of speeding vehicles to obtain the vehicle information of the speeding vehicle images; Construct a speeding vehicle trajectory prediction model for predicting the driving trajectory of speeding vehicles. Combine the extracted vehicle information of the speeding vehicle and the driving trajectory of the speeding vehicle to record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0023] It should be noted that the speeding radar continuously emits electromagnetic waves and receives the reflected signals. By calculating the frequency change of the reflected signals, the radar can monitor the speed of vehicles on the road in real time. When the radar detects that the vehicle speed exceeds the set speed limit value, it will immediately trigger the speeding determination mechanism. At this time, the high-speed camera will start to take pictures of the speeding vehicle. Among them, the speeding determination mechanism judges whether the real-time vehicle speed value is greater than the speed limit threshold. The frequency change of the reflected signal is greater than the preset value, which proves that the signal reflection rate is relatively fast, thus proving that the real-time vehicle speed value is greater than the speed limit value, and it is determined that the vehicle is speeding.

[0024] Furthermore, in a preferred embodiment of the present invention, the S204 is specifically: In the qualified construction site vehicle management system, introduce a motion blur elimination algorithm to eliminate the motion blur of the speeding vehicle images, and perform wavelet denoising processing on the motion blur-eliminated speeding vehicle images to obtain denoised speeding vehicle images; Extract the RGB values of all pixel points in the denoised speeding vehicle images, calculate the proportion of different RGB values, obtain a color dictionary in the big data network, and import the proportion of different RGB values into the color dictionary for retrieval and comparison to obtain the main color of the speeding vehicle; Extract the pixel points at the headlights, intake grilles, and vehicle logos of the denoised speeding vehicle images, and introduce the SIFT feature matching algorithm to retrieve the pixel points at the headlights, intake grilles, and vehicle logos of all brand model vehicles in the big data network, and perform feature matching based on the SIFT feature matching algorithm with the pixel points at the headlights, intake grilles, and vehicle logos of the denoised speeding vehicle images to generate the matching rates between the speeding vehicle and all brand model vehicles; Analyze the matching rates between the speeding vehicle and all brand model vehicles, and calibrate the brand model of the vehicle with the highest matching rate as the brand model of the speeding vehicle; Introduce the Sobel operator to extract the image features of the speeding vehicle license plate position, and perform character analysis on the image features of the speeding vehicle license plate position to determine the license plate number of the speeding vehicle; Classify the main color, brand model, and license plate number of the speeding vehicle to generate the vehicle information of the speeding vehicle.

[0025] It should be noted that since the vehicle is photographed during movement, it is particularly important to analyze the vehicle information after eliminating the motion, so that the vehicle information can be more accurately located. Motion blur elimination algorithms include but are not limited to the BM3D method, which is suitable for capturing vehicles at high speeds. After obtaining the image, filtering processing is still required. The wavelet filtering method can remove the noise on the image to make the image clearer. The vehicle has a main body color, such as the doors, fenders, and hoods of the vehicle should be of a uniform color, and the vehicle has its brand model and license plate number. The vehicle information can be obtained based on the main body color of the vehicle and the brand model and license plate number. First, determine the main body color. The color dictionary records the names corresponding to different colors. The headlights, air intake grilles, and vehicle logos in the noise-reduced speeding vehicle image are usually important signs for judging the brand model of the vehicle. The SIFT feature matching algorithm is a feature data matching algorithm. Different feature data are matched through the SIFT feature matching algorithm to calculate the similarity between the data. If the similarity is higher than the preset value, the vehicle brand model can be generated. The Sobel operator first verifies the vertical edge density of the license plate and performs character segmentation. Finally, the hidden Markov model is used to correct the characters that are easily confused during character extraction to obtain the license plate number. The main body color, brand model and license plate number of the speeding vehicle are classified to generate vehicle information of the speeding vehicle, which is used to build the real-time status of the speeding vehicle and realize vehicle speeding management.

[0026] Furthermore, in a preferred embodiment of the present invention, the step S206 is specifically as follows: Obtain the GPS latitude and longitude of the target high-speed camera, record the real-time speed of the speeding vehicle, and obtain all roads connected to the main road of the construction site and mark them as the roads to be analyzed; The connection relationship of all roads to be analyzed and the main roads at the construction site is encoded through the graph neural network to generate a road topology map and obtain the real-time traffic light status of all roads to be analyzed and the main roads at the construction site; A multimodal spatiotemporal fusion blank model is introduced, and the road topology map, the GPS longitude and latitude of the target high-speed camera, the real-time speed value of the speeding vehicle, the real-time traffic light status of all roads to be analyzed and the main road of the construction site are converted into feature data of the time series feature state, and calibrated as the target time series feature data; In the multimodal space-time fusion blank model, the target time series feature data is subjected to adaptive Kalman filtering to eliminate feature data noise and obtain target filtered time series feature data. At the same time, spatial interaction modeling is performed on the target filtered time series feature data in the multimodal space-time fusion blank model to obtain a speeding vehicle trajectory prediction model. Based on the speeding vehicle trajectory prediction model, all modal trajectories of the speeding vehicle are generated; The Gaussian mixture algorithm is introduced to calculate the distribution weights of speeding vehicles on different modal trajectories. Among them, the distribution weights of different modal trajectories represent the probability values of different driving trajectories of speeding vehicles. Obtain the modal trajectory corresponding to the maximum distribution weight, and obtain the target driving trajectory of the speeding vehicle. Combine the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle to generate and record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0027] It should be noted that constructing a speeding vehicle trajectory prediction model can judge and predict the next driving direction of the speeding vehicle, and generate and record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system according to the next driving direction of the speeding vehicle and the vehicle information in the management system, for speeding management of vehicles, such as punishing speeding vehicles or intercepting speeding vehicles. If it is necessary to determine the vehicle driving trajectory, it is necessary to construct a topological map of all roads connected to the main road of the construction road for predicting the driving trajectory of speeding vehicles in the next period of time. First, coordinate determination is performed to locate the real-time position of the vehicle, and target time-series feature data is obtained for constructing a multi-modal spatio-temporal fusion model, which is a deep learning model for predicting the vehicle's direction. The data needs to be adaptively filtered by the Kalman filter to dynamically adjust the process noise and achieve the purpose of improving data accuracy. The feature data is encoded data in the model. Through the graph attention network, the encoded data can interact to generate an adversarial network. Multiple candidate vehicle trajectories are output in the adversarial network. By judging the rationality of the trajectory, such as conforming to physical laws and the road topological map, all possible vehicle trajectories can be generated. The Gaussian mixture algorithm is an algorithm for calculating the probability values of different vehicle trajectories, from which the target driving trajectory of the speeding vehicle is obtained. Combine the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle to generate and record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0028] As Figure 3 shown, the second aspect of the present invention also provides a vehicle speeding management system based on multi-modal fusion. The vehicle speeding management system includes a memory 31 and a processor 32. The vehicle speeding management method is stored in the memory 31. When the vehicle speeding management method is executed by the processor 32, the following steps are implemented: Perform real-time monitoring of the traffic flow on the main road of the construction site, and determine the installation positions of the speeding radar and the high-speed camera in combination with the real-time monitoring results of the traffic flow; Conduct information interconnection optimization tests on the target speeding radar and the target high-speed camera to realize the construction of a qualified construction site vehicle management system; Run a qualified construction site vehicle management system and combine it with a multi-modal data fusion analysis method to record and display the vehicle dynamics of speeding vehicles in the qualified construction site vehicle management system.

[0029] The above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A vehicle speeding management method based on multimodal fusion, characterized in that, It includes the following steps: S102: Conduct real-time monitoring of vehicle flow on the main road of the construction site, and determine the installation positions of speed radars and high-speed cameras based on the results of the real-time vehicle flow monitoring; S104: Conduct information interconnection optimization tests on the target speed radars and target high-speed cameras to build a qualified construction site vehicle management system; S106: Run the qualified construction site vehicle management system, and record and display the vehicle dynamics of speeding vehicles in the qualified construction site vehicle management system in combination with the multi-modal data fusion analysis method.

2. The vehicle speeding management method based on multi-modal fusion according to claim 1, wherein, The S102 is specifically as follows: Determine the main road of the construction site, introduce the historical data network, and based on the historical data network, determine all the positions on the main road of the construction site where speed radars and high-speed cameras can be installed, and mark them as speed radar installable positions and high-speed camera installable positions; Determine the working ranges of different speed radar installable positions and high-speed camera installable positions on the main road of the construction site, and retrieve the historical vehicle flow in the corresponding working ranges in the historical data network; Preset the target vehicle flow, and divide the speed radar installable positions and high-speed camera installable positions corresponding to the historical vehicle flow not less than the target vehicle flow in the working ranges on the main road of the construction site into target speed radar installation positions and target high-speed camera installation positions; Install speed radars and high-speed cameras at the target speed radar installation positions and target high-speed camera installation positions to obtain the target speed radars and target high-speed cameras.

3. The vehicle speeding management method based on multimodal fusion according to claim 1, wherein The S104 is specifically as follows: Use the same communication protocol for the target speed radars and target high-speed cameras, and mark it as the target communication protocol, where the target communication protocol ensures the same-frequency information interconnection of the target speed radars and target high-speed cameras; Conduct test operations on the target speed radars and target high-speed cameras, and synchronize the data collection times of the target speed radars and target high-speed cameras during the test operations in combination with the target communication protocol; When the data collection times of the target speed radars and target high-speed cameras are synchronized during the test operations, obtain the management system of the main road of the construction site, and perform data interconnection between the target speed radars and target high-speed cameras and the management system of the main road of the construction site to build a construction site vehicle management system; In the construction site vehicle management system, control the target speed radar to output a speed radar test data packet during the test operation, where the speed radar test data packet contains the real-time test speed value and the corresponding timestamp of the construction site vehicle, and after testing and outputting the speed radar test data packet, the response time of the target high-speed camera to generate a capture instruction is marked as the test response time; Preset the standard test response time threshold. If the test response time is maintained within the standard test response time threshold, it is determined that the information interconnection effect of the target speed radars and target high-speed cameras is qualified, and the construction site vehicle management system is divided into a qualified construction site vehicle management system. If the test response time does not remain within the standard test response time threshold, then introduce time-sensitive network technology in the construction site vehicle management system to improve the time synchronization accuracy of the target speed radar and the target high-speed camera until the test response time remains within the standard test response time threshold.

4. The vehicle speeding management method based on multimodal fusion according to claim 1, characterized in that, The S106 is specifically as follows: Run a qualified construction site vehicle management system, based on the target speed radar, emit and receive reflected electromagnetic waves on the main road of the construction site in real time, and at the same time calculate the frequency of the reflected electromagnetic waves in the qualified construction site vehicle management system to generate the real-time speed values of different vehicles on the main road of the construction site. Preset a speed limit threshold in the qualified construction site vehicle management system. If there is no real-time speed value of a vehicle greater than the speed limit threshold, then mark all vehicles passing through the target speed radar as non-speeding vehicles in the qualified construction site vehicle management system and perform storage processing at the same time. When there is a real-time speed value of a vehicle greater than the speed limit threshold, mark the corresponding vehicle as a speeding vehicle, and at the same time start the target high-speed camera in the qualified construction site vehicle management system, and control the target high-speed camera to capture the image of the speeding vehicle in real time, and calibrate it as a speeding vehicle image. Perform image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain the vehicle information of the speeding vehicle image. Construct a speeding vehicle trajectory prediction model for predicting the driving trajectory of the speeding vehicle, and combine the extracted vehicle information of the speeding vehicle and the driving trajectory of the speeding vehicle to record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

5. The vehicle speeding management method based on multimodal fusion according to claim 4, wherein The performing image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain the vehicle information of the speeding vehicle image is specifically as follows: In the qualified construction site vehicle management system, introduce a motion blur elimination algorithm to eliminate the motion blur of the speeding vehicle image, and perform wavelet denoising processing on the speeding vehicle image after motion blur elimination to obtain a denoised speeding vehicle image. Extract the RGB values of all pixel points in the denoised speeding vehicle image, calculate the proportion of different RGB values, obtain a color dictionary in the big data network, and import the proportion of different RGB values into the color dictionary for retrieval and comparison to obtain the main color of the speeding vehicle. Extract the pixel points at the headlights, intake grille, and vehicle logo of the denoised speeding vehicle image, and introduce the SIFT feature matching algorithm. Retrieve the pixel points at the headlights, intake grille, and vehicle logo of all brand model vehicles in the big data network, and perform feature matching based on the SIFT feature matching algorithm with the pixel points at the headlights, intake grille, and vehicle logo of the denoised speeding vehicle image to generate the matching rate between the speeding vehicle and all brand model vehicles. Analyze the matching rate between the speeding vehicle and all brand model vehicles, and calibrate the brand model of the vehicle with the highest matching rate as the brand model of the speeding vehicle. Introduce the Sobel operator to extract the image features of the speeding vehicle license plate position, and perform character analysis on the image features of the speeding vehicle license plate position to determine the license plate number of the speeding vehicle. Classify the main color, brand model, and license plate number of the speeding vehicle to generate vehicle information of the speeding vehicle.

6. The vehicle speeding management method based on multi-modal fusion according to claim 4, wherein, Construct a speeding vehicle trajectory prediction model for predicting the driving trajectory of the speeding vehicle. Combine the extracted vehicle information of the speeding vehicle and the driving trajectory of the speeding vehicle, and record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system. Specifically: Obtain the GPS longitude and latitude of the target highway camera, record the real-time speed value of the speeding vehicle, and obtain all the roads connecting the main roads of the construction site, which are marked as roads to be analyzed. Encode the connection relationships of all the roads to be analyzed and the main roads of the construction site through a graph neural network to generate a road topology map, and obtain the real-time traffic light states of all the roads to be analyzed and the main roads of the construction site. Introduce a multi-modal spatio-temporal fusion blank model, and convert the road topology map, the GPS longitude and latitude of the target highway camera, the real-time speed value of the speeding vehicle, the real-time traffic light states of all the roads to be analyzed and the main roads of the construction site into feature data in a time-series feature state, which is marked as target time-series feature data. In the multi-modal spatio-temporal fusion blank model, perform adaptive Kalman filtering on the target time-series feature data to eliminate the feature data noise and obtain the target filtered time-series feature data. At the same time, perform spatial interaction modeling on the target filtered time-series feature data in the multi-modal spatio-temporal fusion blank model to obtain the speeding vehicle trajectory prediction model. Based on the speeding vehicle trajectory prediction model, generate all modal trajectories of the speeding vehicle. Introduce the Gaussian mixture algorithm to calculate the distribution weights of the speeding vehicle in different modal trajectories. Among them, the distribution weights of different modal trajectories represent the probability values of different driving trajectories of the speeding vehicle. Obtain the modal trajectory corresponding to the maximum distribution weight and get the target driving trajectory of the speeding vehicle. Combine the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle, and generate and record the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

7. A vehicle speeding management system based on multimodal fusion, characterized in that, The vehicle speeding management system includes a memory and a processor. The vehicle speeding management method program is stored in the memory. When the vehicle speeding management method program is executed by the processor, the steps of the vehicle speeding management method described in any one of claims 1-6 are implemented.

Images