Vehicle speeding management method and system based on multimodal fusion

Abstract

The present invention relates to the field of vehicle control and management, and discloses a vehicle speeding management method and system based on multimodal fusion. The method comprises the following steps: monitoring traffic flow on the main roads of a construction site, recording speeding vehicles, and implementing speeding management on the construction site vehicle management system by combining a multimodal data fusion analysis method. Compared to single-modal recognition technology, multimodal fusion technology can fully utilize the complementarity and redundancy between different modal data to improve the accuracy and comprehensiveness of information expression. Furthermore, the technology can adapt to different environments and scenarios, demonstrating greater adaptability and flexibility.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control, and in particular to a vehicle speeding management method and system based on multimodal fusion. Background Art

[0002] Large-scale construction cluster projects are characterized by large construction sites, numerous temporary roads, and a concentration of vehicles on-site, making management challenging. This is particularly true given the large number of vehicles within the multiple construction zones, each with numerous intersections on the main and branch roads. This leads to a significant speeding problem. Due to dust and other issues at the construction site, the license plates of incoming vehicles are often obscured to a certain extent. Managing speeding is a pressing issue for large-scale construction sites. This proposed solution primarily involves speeding radar, high-speed cameras, and a construction site vehicle management system. The speeding radar monitors vehicle speeds in real time by calculating the frequency changes in reflected signals. When the high-speed camera detects a vehicle exceeding the set speed limit, it immediately triggers a speeding detection mechanism, activating the camera and taking a photo of the speeding vehicle. The construction site vehicle management system primarily records basic vehicle information such as vehicle color, vehicle model, license plate, and engine noise, and uses multimodal fusion to manage speeding. Therefore, a vehicle speeding management method and system based on multimodal fusion is proposed. Summary of the Invention

[0003] The present invention overcomes the deficiencies of the prior art and provides a vehicle speeding management method and system based on multimodal fusion.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is:

[0005] A first aspect of the present invention provides a vehicle speeding management method based on multimodal fusion, comprising the following steps:

[0006] S102: Carry out real-time monitoring of traffic flow on the main roads of the construction site, and determine the installation locations of speeding radars and high-speed cameras based on the real-time traffic flow monitoring results;

[0007] S104: Conduct information interconnection optimization tests on target speeding radars and target high-speed cameras to build a qualified construction site vehicle management system;

[0008] S106: Running the qualified construction site vehicle management system, and combining the multimodal data fusion analysis method, recording and displaying the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0009] Furthermore, in a preferred embodiment of the present invention, the step S102 is specifically as follows:

[0010] Determine the main roads at the construction site and introduce a historical data network. Based on the historical data network, determine all locations on the main roads at the construction site where speeding radars and speed cameras can be installed, and mark them as speeding radar installation locations and speed camera installation locations.

[0011] Determine the working ranges of different speed radar installation locations and speed camera installation locations on the main roads of the construction site, and retrieve historical traffic flow within the corresponding working ranges from the historical data network;

[0012] A target traffic volume is preset, and speed radar installation locations and speed camera installation locations corresponding to locations where the historical traffic volume is not less than the target traffic volume within the working range on the main road of the construction site are divided into target speed radar installation locations and target speed camera installation locations;

[0013] An overspeed radar and a high-speed camera are installed correspondingly at the target overspeed radar installation position and the target high-speed camera installation position to obtain a target overspeed radar and a target high-speed camera.

[0014] Furthermore, in a preferred embodiment of the present invention, the step S104 is specifically as follows:

[0015] The target speeding radar and the target high-speed camera use the same communication protocol, which is calibrated as the target communication protocol, wherein the target communication protocol ensures that the target speeding radar and the target high-speed camera achieve information interconnection at the same frequency;

[0016] Conduct test operations on target speeding radars and target high-speed cameras, and synchronize the data collection times of the target speeding radars and target high-speed cameras during the test operations in conjunction with target communication protocols;

[0017] When the target speeding radar and the target high-speed camera collect data at the same time during the test operation, the management system of the main road of the construction site is obtained, and the target speeding radar and the target high-speed camera are interconnected with the management system of the main road of the construction site to build a construction site vehicle management system;

[0018] In the construction site vehicle management system, the target speeding radar is controlled to output a speeding radar test data packet during the test operation, wherein the speeding radar test data packet includes the real-time test speed value of the construction site vehicle and the corresponding timestamp. The response time of the target high-speed camera generating a capture instruction after the speeding radar test data packet is output is tested and calibrated as the test response time;

[0019] A standard test response time threshold is preset. If the test response time remains within the standard test response time threshold, the information interconnection effect of the target speeding radar and the target high-speed camera is determined to be qualified, and the construction site vehicle management system is classified as a qualified construction site vehicle management system;

[0020] If the test response time is not maintained within the standard test response time threshold, time-sensitive networking technology will be introduced into the construction site vehicle management system to improve the time synchronization accuracy of the target speeding radar and the target high-speed camera until the test response time is maintained within the standard test response time threshold.

[0021] Furthermore, in a preferred embodiment of the present invention, the step S106 is specifically as follows:

[0022] Run a qualified construction site vehicle management system that uses a target speeding radar to transmit and receive reflected electromagnetic waves on the main roads of the construction site in real time. At the same time, the frequency of the reflected electromagnetic waves is calculated within the qualified construction site vehicle management system to generate real-time speed values ​​for different vehicles on the main roads of the construction site.

[0023] A speed limit threshold is preset in the qualified construction site vehicle management system. If no vehicle has a real-time speed greater than the speed limit threshold, all vehicles passing through the target speeding radar are marked as not speeding in the qualified construction site vehicle management system and stored for processing.

[0024] When the real-time speed value of a vehicle exceeds the speed limit threshold, the corresponding vehicle will be marked as a speeding vehicle. At the same time, a target high-speed camera will be activated in the qualified construction site vehicle management system, and the target high-speed camera will be controlled to capture an image of the speeding vehicle in real time, which will be marked as an image of the speeding vehicle.

[0025] Performing image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain vehicle information of the speeding vehicle image;

[0026] A speeding vehicle trajectory prediction model is constructed to predict the driving trajectory of speeding vehicles. The vehicle dynamics of speeding vehicles are recorded in the qualified construction site vehicle management system by combining the extracted vehicle information of speeding vehicles and the driving trajectory of speeding vehicles.

[0027] Furthermore, in a preferred embodiment of the present invention, the image recognition, image feature extraction, and image feature classification are performed on the speeding vehicle image to obtain vehicle information of the speeding vehicle image, specifically:

[0028] In the qualified construction site vehicle management system, a motion blur removal algorithm is introduced to remove motion blur from the speeding vehicle image, and the speeding vehicle image after motion blur removal is processed by wavelet noise reduction to obtain a denoised speeding vehicle image.

[0029] Extract the RGB values ​​of all pixels in the denoised speeding vehicle image and calculate the proportions of different RGB values. Then, obtain a color dictionary from the big data network and import the proportions of different RGB values ​​into the color dictionary for search and comparison to obtain the main color of the speeding vehicle.

[0030] Extract the pixels of the headlights, grilles, and vehicle logos in the noise-reduced speeding vehicle image, and introduce the SIFT feature matching algorithm to retrieve the pixels of the headlights, grilles, and vehicle logos of all brands and models of vehicles in the big data network. Based on the SIFT feature matching algorithm, feature matching is performed with the pixels of the headlights, grilles, and vehicle logos in the noise-reduced speeding vehicle image to generate the matching rate between the speeding vehicle and all brands and models of vehicles.

[0031] Analyze the matching rate between the speeding vehicle and all brands and models of vehicles, and mark the brand and model of the vehicle with the highest matching rate as the brand and model of the speeding vehicle;

[0032] The Sobel operator is introduced to extract the image features of the license plate position of the speeding vehicle, and the character analysis is performed on the image features of the license plate position of the speeding vehicle to determine the license plate number of the speeding vehicle;

[0033] The body color, brand model and license plate number of the speeding vehicle are classified to generate vehicle information of the speeding vehicle.

[0034] Furthermore, in a preferred embodiment of the present invention, the speeding vehicle trajectory prediction model is constructed to predict the driving trajectory of the speeding vehicle, and the vehicle dynamics of the speeding vehicle are recorded in the qualified construction site vehicle management system in combination with the extracted vehicle information of the speeding vehicle and the driving trajectory of the speeding vehicle, specifically:

[0035] Obtain the GPS latitude and longitude of the target high-speed camera, record the real-time speed of speeding vehicles, and obtain all roads connected to the main road of the construction site and mark them as roads to be analyzed;

[0036] The graph neural network is used to encode the connection relationship between all roads to be analyzed and the main roads at the construction site, generate a road topology map, and obtain the real-time traffic light status of all roads to be analyzed and the main roads at the construction site;

[0037] A multimodal spatiotemporal fusion blank model is introduced, and the road topology map, the GPS latitude and longitude of the target high-speed camera, the real-time speed value of the speeding vehicle, the real-time traffic light status of all roads to be analyzed and the main road of the construction site are converted into feature data of the time series feature state and calibrated as the target time series feature data;

[0038] In the multimodal spatiotemporal fusion blank model, adaptive Kalman filtering is performed on the target time series feature data to eliminate feature data noise and obtain target filtered time series feature data. At the same time, spatial interaction modeling is performed on the target filtered time series feature data in the multimodal spatiotemporal fusion blank model to obtain a speeding vehicle trajectory prediction model.

[0039] generating all modal trajectories of the speeding vehicle based on the speeding vehicle trajectory prediction model;

[0040] A Gaussian mixture algorithm is introduced to calculate the distribution weights of speeding vehicles in different modal trajectories, where the distribution weights of different modal trajectories represent the probability values ​​of different speeding vehicle trajectories.

[0041] The modal trajectory corresponding to the maximum distribution weight is obtained, and the target driving trajectory of the speeding vehicle is obtained. Combined with the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle, the vehicle dynamics of the speeding vehicle are generated and recorded in the qualified construction site vehicle management system.

[0042] A second aspect of the present invention further provides a vehicle speeding management system based on multimodal fusion, the vehicle speeding management system comprising a memory and a processor, the memory storing a vehicle speeding management method, and the vehicle speeding management method, when executed by the processor, implementing the following steps:

[0043] Carry out real-time monitoring of traffic flow on the main roads of the construction site, and determine the installation locations of speeding radars and high-speed cameras based on the real-time monitoring results;

[0044] Conduct information interconnection optimization tests on target speeding radars and target high-speed cameras to build a qualified construction site vehicle management system;

[0045] Run a qualified construction site vehicle management system and combine it with multimodal data fusion analysis methods to record and display the vehicle dynamics of speeding vehicles in the qualified construction site vehicle management system.

[0046] This invention addresses the technical deficiencies in the background art and has the following beneficial effects: monitoring traffic flow on construction site main roads, recording speeding vehicles, and integrating multimodal data fusion analysis methods into the construction site vehicle management system to manage speeding. Compared to single-modality recognition technologies, multimodal fusion technology can fully utilize the complementarity and redundancy between different modal data, improving the accuracy and comprehensiveness of information expression. Furthermore, this technology can adapt to different environments and scenarios, offering greater adaptability and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying any creative work, they can also obtain drawings of other embodiments based on these drawings.

[0048] Figure 1 A flow chart of a vehicle speeding management method based on multimodal fusion is shown;

[0049] Figure 2 A flow chart of a method for recording and displaying the vehicle dynamics of speeding vehicles in a qualified construction site vehicle management system in combination with a multimodal data fusion analysis method is shown;

[0050] Figure 3 A procedural view of a vehicle speeding management system based on multimodal fusion is shown. DETAILED DESCRIPTION

[0051] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0053] Figure 1 A flow chart of a vehicle speeding management method based on multimodal fusion is shown, comprising the following steps:

[0054] S102: Carry out real-time monitoring of traffic flow on the main roads of the construction site, and determine the installation locations of speeding radars and high-speed cameras based on the real-time traffic flow monitoring results;

[0055] S104: Conduct information interconnection optimization tests on target speeding radars and target high-speed cameras to build a qualified construction site vehicle management system;

[0056] S106: Running the qualified construction site vehicle management system, and combining the multimodal data fusion analysis method, recording and displaying the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system.

[0057] Furthermore, in a preferred embodiment of the present invention, the step S102 is specifically as follows:

[0058] Determine the main roads at the construction site and introduce a historical data network. Based on the historical data network, determine all locations on the main roads at the construction site where speeding radars and speed cameras can be installed, and mark them as speeding radar installation locations and speed camera installation locations.

[0059] Determine the working ranges of different speed radar installation locations and speed camera installation locations on the main roads of the construction site, and retrieve historical traffic flow within the corresponding working ranges from the historical data network;

[0060] A target traffic volume is preset, and speed radar installation locations and speed camera installation locations corresponding to locations where the historical traffic volume is not less than the target traffic volume within the working range on the main road of the construction site are divided into target speed radar installation locations and target speed camera installation locations;

[0061] An overspeed radar and a high-speed camera are installed correspondingly at the target overspeed radar installation position and the target high-speed camera installation position to obtain a target overspeed radar and a target high-speed camera.

[0062] It should be noted that large-scale construction cluster projects are characterized by large construction sites, numerous temporary roads, and a concentration of vehicles on-site, making management difficult. This is particularly true given the large number of vehicles in each of the multiple sub-construction areas, as well as the frequent intersection of vehicles on the main and branch roads of the construction site. This leads to a serious speeding problem. To prevent dangerous speeding incidents, speeding radars are required to monitor for speeding, and high-speed cameras are required to confirm vehicle information, providing alerts and penalties for speeding. First, the installation locations for the speeding radars and high-speed cameras must be determined to ensure they can record and capture all vehicles on the main roads of the construction site. The installation locations can be determined based on traffic flow, and once the locations are determined, the speeding radars and cameras must be installed.

[0063] Furthermore, in a preferred embodiment of the present invention, the step S104 is specifically as follows:

[0064] The target speeding radar and the target high-speed camera use the same communication protocol, which is calibrated as the target communication protocol, wherein the target communication protocol ensures that the target speeding radar and the target high-speed camera achieve information interconnection at the same frequency;

[0065] Conduct test operations on target speeding radars and target high-speed cameras, and synchronize the data collection times of the target speeding radars and target high-speed cameras during the test operations in conjunction with target communication protocols;

[0066] When the target speeding radar and the target high-speed camera collect data at the same time during the test operation, the management system of the main road of the construction site is obtained, and the target speeding radar and the target high-speed camera are interconnected with the management system of the main road of the construction site to build a construction site vehicle management system;

[0067] In the construction site vehicle management system, the target speeding radar is controlled to output a speeding radar test data packet during the test operation, wherein the speeding radar test data packet includes the real-time test speed value of the construction site vehicle and the corresponding timestamp. The response time of the target high-speed camera generating a capture instruction after the speeding radar test data packet is output is tested and calibrated as the test response time;

[0068] A standard test response time threshold is preset. If the test response time remains within the standard test response time threshold, the information interconnection effect of the target speeding radar and the target high-speed camera is determined to be qualified, and the construction site vehicle management system is classified as a qualified construction site vehicle management system;

[0069] If the test response time is not maintained within the standard test response time threshold, time-sensitive networking technology will be introduced into the construction site vehicle management system to improve the time synchronization accuracy of the target speeding radar and the target high-speed camera until the test response time is maintained within the standard test response time threshold.

[0070] It should be noted that the target speeding radar and target highway camera achieve synchronous information interconnection. This ensures that the speeding camera can capture images immediately after the speeding radar detects a speeding vehicle. Otherwise, if a vehicle passes the speeding camera, it will not be able to clearly detect or even detect the speeding vehicle. The target speeding radar and target highway camera ensure that the data collection time is synchronized during testing. A construction site vehicle management system is established to record basic vehicle information such as color, model, and license plate of incoming vehicles, and to manage speeding vehicles based on multimodal fusion methods. Time-sensitive networking technology is an algorithm that improves the time synchronization accuracy of the target speeding radar and target highway camera, and is used to improve the data sensitivity of the time synchronization accuracy of the target speeding radar and target highway camera. The target communication protocol uses the PTP light communication protocol to achieve microsecond-level time synchronization.

[0071] Figure 2 A flow chart of a method for recording and displaying the vehicle dynamics of speeding vehicles in a qualified construction site vehicle management system by combining a multimodal data fusion analysis method is shown, including the following steps:

[0072] S202: running the qualified construction site vehicle management system and, in combination with a multimodal data fusion analysis method, recording and displaying the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system;

[0073] S204: performing image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain vehicle information of the speeding vehicle image;

[0074] S206: Constructing a speeding vehicle trajectory prediction model to predict the speeding vehicle's driving trajectory, combining the extracted vehicle information of the speeding vehicle and the speeding vehicle's driving trajectory, and recording the speeding vehicle's vehicle dynamics in the qualified construction site vehicle management system.

[0075] Furthermore, in a preferred embodiment of the present invention, the step S106 is specifically as follows:

[0076] Run a qualified construction site vehicle management system that uses a target speeding radar to transmit and receive reflected electromagnetic waves on the main roads of the construction site in real time. At the same time, the frequency of the reflected electromagnetic waves is calculated within the qualified construction site vehicle management system to generate real-time speed values ​​for different vehicles on the main roads of the construction site.

[0077] A speed limit threshold is preset in the qualified construction site vehicle management system. If no vehicle has a real-time speed greater than the speed limit threshold, all vehicles passing through the target speeding radar are marked as not speeding in the qualified construction site vehicle management system and stored for processing.

[0078] When the real-time speed value of a vehicle exceeds the speed limit threshold, the corresponding vehicle will be marked as a speeding vehicle. At the same time, a target high-speed camera will be activated in the qualified construction site vehicle management system, and the target high-speed camera will be controlled to capture an image of the speeding vehicle in real time, which will be marked as an image of the speeding vehicle.

[0079] Performing image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain vehicle information of the speeding vehicle image;

[0080] A speeding vehicle trajectory prediction model is constructed to predict the driving trajectory of speeding vehicles. The vehicle dynamics of speeding vehicles are recorded in the qualified construction site vehicle management system by combining the extracted vehicle information of speeding vehicles and the driving trajectory of speeding vehicles.

[0081] It should be noted that speeding radar continuously emits electromagnetic waves and receives reflected signals. By calculating the frequency changes in the reflected signals, the radar monitors the speed of vehicles on the road in real time. When the radar detects that a vehicle's speed exceeds the set speed limit, it immediately triggers the speeding detection mechanism. At this point, the high-speed camera activates and takes a photo of the speeding vehicle. The speeding detection mechanism then determines whether the vehicle's real-time speed exceeds the speed limit threshold. If the frequency change in the reflected signal exceeds the preset value, it indicates that the signal reflection rate is high, thus proving that the vehicle's real-time speed exceeds the speed limit, indicating that the vehicle is speeding.

[0082] Furthermore, in a preferred embodiment of the present invention, the step S204 is specifically as follows:

[0083] In the qualified construction site vehicle management system, a motion blur removal algorithm is introduced to remove motion blur from the speeding vehicle image, and the speeding vehicle image after motion blur removal is processed by wavelet noise reduction to obtain a denoised speeding vehicle image.

[0084] Extract the RGB values ​​of all pixels in the denoised speeding vehicle image and calculate the proportions of different RGB values. Then, obtain a color dictionary from the big data network and import the proportions of different RGB values ​​into the color dictionary for search and comparison to obtain the main color of the speeding vehicle.

[0085] Extract the pixels of the headlights, grilles, and vehicle logos in the noise-reduced speeding vehicle image, and introduce the SIFT feature matching algorithm to retrieve the pixels of the headlights, grilles, and vehicle logos of all brands and models of vehicles in the big data network. Based on the SIFT feature matching algorithm, feature matching is performed with the pixels of the headlights, grilles, and vehicle logos in the noise-reduced speeding vehicle image to generate the matching rate between the speeding vehicle and all brands and models of vehicles.

[0086] Analyze the matching rate between the speeding vehicle and all brands and models of vehicles, and mark the brand and model of the vehicle with the highest matching rate as the brand and model of the speeding vehicle;

[0087] The Sobel operator is introduced to extract the image features of the license plate position of the speeding vehicle, and the character analysis is performed on the image features of the license plate position of the speeding vehicle to determine the license plate number of the speeding vehicle;

[0088] The body color, brand model and license plate number of the speeding vehicle are classified to generate vehicle information of the speeding vehicle.

[0089] It's important to note that since vehicles are captured while in motion, analyzing vehicle information after motion reduction is particularly important for more accurate positioning. Motion blur reduction algorithms include, but are not limited to, the BM3D method, which is suitable for capturing vehicles at higher speeds. After obtaining the image, filtering is still required. Wavelet filtering can remove noise and improve image clarity. Vehicles have a main body color, such as doors, fenders, and hoods, which should be a uniform color. They also have their make, model, and license plate number. Vehicle information can be derived based on the main body color, make, model, and license plate number. First, determine the main body color. A color dictionary lists the names corresponding to different colors. In de-noised speeding vehicle images, the headlights, grille, and vehicle logo are often important indicators for determining the vehicle's make and model. The SIFT feature matching algorithm is a feature data matching algorithm. It calculates the similarity between different feature data. If the similarity exceeds a preset value, the vehicle's make and model can be determined. The Sobel operator first verifies the vertical edge density of the license plate and performs character segmentation. Finally, a hidden Markov model is used to correct for easily confused characters during character extraction to obtain the license plate number. The speeding vehicle's body color, make, model, and license plate number are categorized to generate vehicle information. This information is used to establish real-time status of speeding vehicles and implement speeding management.

[0090] Furthermore, in a preferred embodiment of the present invention, the step S206 is specifically as follows:

[0091] Obtain the GPS latitude and longitude of the target high-speed camera, record the real-time speed of speeding vehicles, and obtain all roads connected to the main road of the construction site and mark them as roads to be analyzed;

[0092] The graph neural network is used to encode the connection relationship between all roads to be analyzed and the main roads at the construction site, generate a road topology map, and obtain the real-time traffic light status of all roads to be analyzed and the main roads at the construction site;

[0093] A multimodal spatiotemporal fusion blank model is introduced, and the road topology map, the GPS latitude and longitude of the target high-speed camera, the real-time speed value of the speeding vehicle, the real-time traffic light status of all roads to be analyzed and the main road of the construction site are converted into feature data of the time series feature state and calibrated as the target time series feature data;

[0094] In the multimodal spatiotemporal fusion blank model, adaptive Kalman filtering is performed on the target time series feature data to eliminate feature data noise and obtain target filtered time series feature data. At the same time, spatial interaction modeling is performed on the target filtered time series feature data in the multimodal spatiotemporal fusion blank model to obtain a speeding vehicle trajectory prediction model.

[0095] generating all modal trajectories of the speeding vehicle based on the speeding vehicle trajectory prediction model;

[0096] A Gaussian mixture algorithm is introduced to calculate the distribution weights of speeding vehicles in different modal trajectories, where the distribution weights of different modal trajectories represent the probability values ​​of different speeding vehicle trajectories.

[0097] The modal trajectory corresponding to the maximum distribution weight is obtained, and the target driving trajectory of the speeding vehicle is obtained. Combined with the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle, the vehicle dynamics of the speeding vehicle are generated and recorded in the qualified construction site vehicle management system.

[0098] It should be noted that the speeding vehicle trajectory prediction model can predict the speeding vehicle's future direction. Based on the speeding vehicle's future direction and vehicle information, the management system generates and records the speeding vehicle's dynamics in the qualified construction site vehicle management system. This is used for speeding management, such as penalizing or stopping the speeding vehicle. To determine the vehicle's trajectory, a topological map of all roads connected to the main construction road is constructed to predict the speeding vehicle's trajectory over a period of time. First, coordinates are determined to locate the vehicle's real-time position, and target time series feature data is obtained to construct a multimodal spatiotemporal fusion model—a deep learning model for predicting vehicle direction. The data is subjected to an adaptive Kalman filter to dynamically adjust for process noise and improve data accuracy. The feature data is encoded within the model. Through a graph attention network, the encoded data can be interacted to generate an adversarial network. The adversarial network outputs multiple candidate vehicle trajectories. By assessing the plausibility of each trajectory, such as its compliance with physical laws and road topology, all possible vehicle trajectories can be generated. The Gaussian mixture algorithm calculates the probability values ​​of different vehicle trajectories, thereby obtaining the target driving trajectory of the speeding vehicle. Combined with the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle, the vehicle dynamics of the speeding vehicle are generated and recorded in the qualified construction site vehicle management system.

[0099] like Figure 3 As shown, the second aspect of the present invention further provides a vehicle speeding management system based on multimodal fusion, the vehicle speeding management system includes a memory 31 and a processor 32, the memory 31 stores a vehicle speeding management method, and when the vehicle speeding management method is executed by the processor 32, the following steps are implemented:

[0100] Carry out real-time monitoring of traffic flow on the main roads of the construction site, and determine the installation locations of speeding radars and high-speed cameras based on the real-time monitoring results;

[0101] Conduct information interconnection optimization tests on target speeding radars and target high-speed cameras to build a qualified construction site vehicle management system;

[0102] Run a qualified construction site vehicle management system and combine it with multimodal data fusion analysis methods to record and display the vehicle dynamics of speeding vehicles in the qualified construction site vehicle management system.

[0103] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A vehicle speeding management method based on multimodal fusion, characterized in that: The following steps are involved: S102: Carry out real-time monitoring of traffic flow on the main roads of the construction site, and determine the installation locations of speeding radars and high-speed cameras based on the real-time traffic flow monitoring results; S104: Conduct information interconnection optimization tests on target speeding radars and target high-speed cameras to build a qualified construction site vehicle management system; S106: Running the qualified construction site vehicle management system and combining it with a multimodal data fusion analysis method to record and display the vehicle dynamics of the speeding vehicle in the qualified construction site vehicle management system; The S106 is specifically as follows: Run a qualified construction site vehicle management system that uses a target speeding radar to transmit and receive reflected electromagnetic waves on the main roads of the construction site in real time. At the same time, the frequency of the reflected electromagnetic waves is calculated within the qualified construction site vehicle management system to generate real-time speed values ​​for different vehicles on the main roads of the construction site. A speed limit threshold is preset in the qualified construction site vehicle management system. If no vehicle has a real-time speed greater than the speed limit threshold, all vehicles passing through the target speeding radar are marked as not speeding in the qualified construction site vehicle management system and stored for processing. When the real-time speed value of a vehicle exceeds the speed limit threshold, the corresponding vehicle will be marked as a speeding vehicle. At the same time, a target high-speed camera will be activated in the qualified construction site vehicle management system, and the target high-speed camera will be controlled to capture an image of the speeding vehicle in real time, which will be marked as an image of the speeding vehicle. Performing image recognition, image feature extraction, and image feature classification on the speeding vehicle image to obtain vehicle information of the speeding vehicle image; Construct a speeding vehicle trajectory prediction model to predict the speeding vehicle's driving trajectory. Combine the extracted vehicle information and the speeding vehicle's driving trajectory to record the speeding vehicle's dynamics in the qualified construction site vehicle management system. The speeding vehicle trajectory prediction model is constructed to predict the speeding vehicle's driving trajectory, and the vehicle dynamics of the speeding vehicle are recorded in the qualified construction site vehicle management system in combination with the extracted vehicle information of the speeding vehicle and the speeding vehicle's driving trajectory, specifically: Obtain the GPS latitude and longitude of the target high-speed camera, record the real-time speed of speeding vehicles, and obtain all roads connected to the main road of the construction site and mark them as roads to be analyzed; The graph neural network is used to encode the connection relationship between all roads to be analyzed and the main roads at the construction site, generate a road topology map, and obtain the real-time traffic light status of all roads to be analyzed and the main roads at the construction site; A multimodal spatiotemporal fusion blank model is introduced, and the road topology map, the GPS latitude and longitude of the target high-speed camera, the real-time speed value of the speeding vehicle, the real-time traffic light status of all roads to be analyzed and the main road of the construction site are converted into feature data of the time series feature state and calibrated as the target time series feature data; In the multimodal spatiotemporal fusion blank model, adaptive Kalman filtering is performed on the target time series feature data to eliminate feature data noise and obtain target filtered time series feature data. At the same time, spatial interaction modeling is performed on the target filtered time series feature data in the multimodal spatiotemporal fusion blank model to obtain a speeding vehicle trajectory prediction model. generating all modal trajectories of the speeding vehicle based on the speeding vehicle trajectory prediction model; A Gaussian mixture algorithm is introduced to calculate the distribution weights of speeding vehicles in different modal trajectories, where the distribution weights of different modal trajectories represent the probability values ​​of different speeding vehicle trajectories. The modal trajectory corresponding to the maximum distribution weight is obtained, and the target driving trajectory of the speeding vehicle is obtained. Combined with the target driving trajectory of the speeding vehicle and the vehicle information of the speeding vehicle, the vehicle dynamics of the speeding vehicle are generated and recorded in the qualified construction site vehicle management system.

2. The vehicle speeding management method based on multimodal fusion according to claim 1 is characterized in that: The S102 is specifically as follows: Determine the main roads at the construction site and introduce a historical data network. Based on the historical data network, determine all locations on the main roads at the construction site where speeding radars and speed cameras can be installed, and mark them as speeding radar installation locations and speed camera installation locations. Determine the working ranges of different speed radar installation locations and speed camera installation locations on the main roads of the construction site, and retrieve historical traffic flow within the corresponding working ranges from the historical data network; A target traffic volume is preset, and speed radar installation locations and speed camera installation locations corresponding to locations where the historical traffic volume is not less than the target traffic volume within the working range on the main road of the construction site are divided into target speed radar installation locations and target speed camera installation locations; An overspeed radar and a high-speed camera are installed correspondingly at the target overspeed radar installation position and the target high-speed camera installation position to obtain a target overspeed radar and a target high-speed camera.

3. The vehicle speeding management method based on multimodal fusion according to claim 1 is characterized in that: The S104 is specifically as follows: The target speeding radar and the target high-speed camera use the same communication protocol, which is calibrated as the target communication protocol, wherein the target communication protocol ensures that the target speeding radar and the target high-speed camera achieve information interconnection at the same frequency; Conduct test operations on target speeding radars and target high-speed cameras, and synchronize the data collection times of the target speeding radars and target high-speed cameras during the test operations in conjunction with target communication protocols; When the target speeding radar and the target high-speed camera collect data at the same time during the test operation, the management system of the main road of the construction site is obtained, and the target speeding radar and the target high-speed camera are interconnected with the management system of the main road of the construction site to build a construction site vehicle management system; In the construction site vehicle management system, the target speeding radar is controlled to output a speeding radar test data packet during the test operation, wherein the speeding radar test data packet includes the real-time test speed value of the construction site vehicle and the corresponding timestamp. The response time of the target high-speed camera generating a capture instruction after the speeding radar test data packet is output is tested and calibrated as the test response time; A standard test response time threshold is preset. If the test response time remains within the standard test response time threshold, the information interconnection effect of the target speeding radar and the target high-speed camera is determined to be qualified, and the construction site vehicle management system is classified as a qualified construction site vehicle management system; If the test response time is not maintained within the standard test response time threshold, time-sensitive networking technology will be introduced into the construction site vehicle management system to improve the time synchronization accuracy of the target speeding radar and the target high-speed camera until the test response time is maintained within the standard test response time threshold.

4. The vehicle speeding management method based on multimodal fusion according to claim 1 is characterized in that: The image recognition, image feature extraction, and image feature classification of the speeding vehicle image are performed to obtain vehicle information of the speeding vehicle image, specifically: In the qualified construction site vehicle management system, a motion blur removal algorithm is introduced to remove motion blur from the speeding vehicle image, and the speeding vehicle image after motion blur removal is processed by wavelet noise reduction to obtain a denoised speeding vehicle image. Extract the RGB values ​​of all pixels in the denoised speeding vehicle image and calculate the proportions of different RGB values. Then, obtain a color dictionary from the big data network and import the proportions of different RGB values ​​into the color dictionary for search and comparison to obtain the main color of the speeding vehicle. Extract the pixels of the headlights, grilles, and logos in the noise-reduced speeding vehicle image, and introduce the SIFT feature matching algorithm to retrieve the pixels of the headlights, grilles, and logos of all brands and models of vehicles in the big data network. Based on the SIFT feature matching algorithm, feature matching is performed with the pixels of the headlights, grilles, and logos in the noise-reduced speeding vehicle image to generate the matching rate between the speeding vehicle and all brands and models of vehicles. Analyze the matching rate between the speeding vehicle and all brands and models of vehicles, and mark the brand and model of the vehicle with the highest matching rate as the brand and model of the speeding vehicle; The Sobel operator is introduced to extract the image features of the license plate position of the speeding vehicle, and the character analysis is performed on the image features of the license plate position of the speeding vehicle to determine the license plate number of the speeding vehicle; The body color, brand model and license plate number of the speeding vehicle are classified to generate vehicle information of the speeding vehicle.

5. The vehicle speeding management system based on multimodal fusion is characterized by: The vehicle overspeed management system includes a memory and a processor, wherein a vehicle overspeed management method program is stored in the memory. When the vehicle overspeed management method program is executed by the processor, the vehicle overspeed management method steps according to any one of claims 1 to 4 are implemented.

Images