A method, device and storage medium for road information early warning

Abstract

This application discloses a road information early warning method, device, and storage medium. The method collects sound signals transmitted through the road surface at the location of the road to be warned, simultaneously performs vehicle presence detection and vehicle type identification using a multi-task convolutional neural network model, and uses optical equipment to detect vehicle arrival. It also determines the current weather condition based on image brightness, contrast, and laser return signal intensity. The method adaptively weights and fuses the confidence levels of sound detection and optical equipment detection based on weather condition factors. When the overall confidence level exceeds a preset threshold, an early warning is triggered, and a three-level differentiated early warning strategy (standard, enhanced, or emergency) is implemented based on the identified vehicle type. The multi-task convolutional neural network model is continuously optimized online through comparison of optical and sound detection results.

Description

Technical Field

[0001] This application relates to the field of road information early warning, specifically to a road information early warning method, device and storage medium. Background Technology

[0002] Road information early warning refers to a service that uses various sensors and information systems to collect and analyze road traffic conditions in real time, and to provide early warnings of potential traffic hazards or congestion. Its purpose is to improve road traffic safety and reduce traffic accident losses.

[0003] Currently, road information early warning methods mainly involve setting up warning devices ahead of the road area requiring warning to alert oncoming vehicles. However, existing solutions have the following shortcomings:

[0004] First, for dangerous locations such as sharp turns on mountain roads or remote sections of roads, construction areas, and other dangerous locations, existing methods rely on optical equipment such as laser emitters or cameras for detection. In foggy, rainy, or other extreme weather conditions, the detection accuracy of optical equipment drops significantly, leading to missed or false alarms.

[0005] Second, existing fusion early warning schemes mostly adopt a simple OR logic of "any sensor detects and triggers an early warning", without considering the difference in confidence of each sensor under different weather conditions, resulting in a high false alarm rate under severe weather conditions.

[0006] Third, the existing early warning scheme only outputs a binary judgment of "whether there is a vehicle" without distinguishing between vehicle types (light, medium, and heavy). However, different types of vehicles have significant differences in inertia and braking distance. Using a uniform early warning strategy cannot meet the actual needs of heavy vehicles, which have long braking distances and require longer warning times.

[0007] In view of the above deficiencies, this application provides a road information early warning method, device and storage medium to solve the above technical problems. Summary of the Invention

[0008] This application provides a road information early warning method, device and storage medium to solve the technical problems of high false alarm rate caused by insufficient consideration of weather factors in the multi-sensor fusion decision-making of existing road information early warning schemes, and the inability of single early warning strategies to adapt to different vehicle types.

[0009] In a first aspect, this application provides a road information early warning method, the method comprising the following steps:

[0010] Step 1: Collect the sound signal transmitted through the road surface at the location of the road to be warned;

[0011] Step 2: Process the sound signal, detect whether a vehicle is driving towards the location of the warning road using a multi-task convolutional neural network model, and at the same time identify the vehicle type of the sound signal to determine whether the driving vehicle is a light vehicle, medium vehicle or heavy vehicle.

[0012] Step 3: Use optical equipment to detect whether any vehicles are driving towards the location of the road to be warned, and determine the weather condition factor based on the current weather conditions;

[0013] Step 4: Based on the weather state factor, assign corresponding weight coefficients to the sound signal detection confidence level in Step 2 and the optical device detection confidence level in Step 3, and calculate the comprehensive detection confidence level; when the comprehensive detection confidence level exceeds a preset threshold, determine that a vehicle is driving towards the location of the road to be warned, and execute differentiated warnings based on the vehicle type identified in Step 2.

[0014] Step 4 further includes: when the optical device in step 3 detects a vehicle moving toward the location of the road to be warned but the overall detection confidence does not exceed the preset threshold, a differentiated warning is still executed, and the sound signal at this time is recorded to optimize the multi-task convolutional neural network model in step 2 online.

[0015] Preferably, in step 2, the multi-task convolutional neural network model includes a shared feature extraction layer and two parallel output heads. The first output head is used for binary classification detection of vehicle presence, and the second output head is used for three-class classification recognition of vehicle type.

[0016] Preferably, the sound signal processing includes: converting the sound signal into a spectrogram through short-time Fourier transform, and then inputting it into the multi-task convolutional neural network model after data augmentation using SpecAugment technology.

[0017] Preferably, in step 3, the current weather condition is determined by comprehensively considering the camera image brightness value, contrast value, and laser return signal intensity, and a corresponding weather condition factor is generated.

[0018] Preferably, in step 4, the formula for calculating the overall detection confidence level is C=α. 声 ×C 声 +α 光 ×C 光 On sunny days α 光 >α 声 α on foggy or rainy days 声 ≥α 光 .

[0019] Preferably, the differentiated warning is divided into three levels: light vehicles trigger the standard warning mode, medium vehicles trigger the enhanced warning mode, and heavy vehicles trigger the emergency warning mode.

[0020] Preferably, when the optical device detects no vehicles, idle road image data is collected at equal time intervals, vehicles in the idle images are detected by the YOLO target detection model, and the detection results are compared with the sound signal to optimize the multi-task convolutional neural network model in step 2.

[0021] Preferably, the location of the road to be warned includes the road ahead of each of the traffic light intersection area, the sharp turn area, and the road construction area.

[0022] Secondly, this application provides a computer device, including a processor and a memory, wherein the computer program, when executed by the processor, implements the above-described road information warning method.

[0023] Compared with the prior art, this application has at least the following beneficial effects:

[0024] (1) An adaptive weighted fusion decision-making mechanism based on weather conditions is introduced. The weight coefficients of sound signal detection and optical equipment detection are dynamically adjusted according to the real-time weather conditions. This avoids false warnings caused by optical equipment false alarms in severe weather due to simple OR logic, effectively reducing the false alarm rate, while ensuring that the advantages of optical equipment detection accuracy are fully utilized under clear weather conditions.

[0025] (2) By designing a dual-output head for a multi-task convolutional neural network model, the vehicle type can be synchronously identified using road vibration sound signals without adding extra hardware. This upgrades the warning method from a single mode to a three-level differentiated warning strategy. Heavy vehicles, due to their large inertia and long braking distance, receive longer warning times and stronger warning intensity, significantly improving the actual safety performance of the warning system.

[0026] (3) By continuously comparing the optical detection results with the sound signal detection results, an online closed-loop optimization mechanism for the multi-task convolutional neural network model was constructed. As the deployment time increases, the sound detection accuracy continues to improve, effectively adapting to the differences in sound characteristics under different road environments. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the workflow of a road information early warning method disclosed in an embodiment of this application. Detailed Implementation

[0028] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0029] This application provides a road information early warning method, the method comprising the following steps:

[0030] Step 1: Collect the sound signal transmitted through the road surface at the location of the road to be warned;

[0031] Step 2: Process the sound signal, detect whether a vehicle is driving towards the location of the warning road using a multi-task convolutional neural network model, and at the same time identify the vehicle type of the sound signal to determine whether the driving vehicle is a light vehicle, medium vehicle or heavy vehicle.

[0032] Step 3: Use optical equipment to detect whether any vehicles are driving towards the location of the road to be warned, and determine the weather condition factor based on the current weather conditions;

[0033] Step 4: Based on the weather state factor, assign corresponding weight coefficients to the sound signal detection confidence level in Step 2 and the optical device detection confidence level in Step 3, and calculate the comprehensive detection confidence level; when the comprehensive detection confidence level exceeds a preset threshold, determine that a vehicle is driving towards the location of the road to be warned, and execute differentiated warnings based on the vehicle type identified in Step 2.

[0034] Step 4 further includes: when the optical device in step 3 detects a vehicle moving toward the location of the road to be warned but the comprehensive detection confidence does not exceed the preset threshold, a differentiated warning is still executed, and the sound signal transmitted through the road surface at this time is recorded and used as training data to optimize the multi-task convolutional neural network model in step 2 online.

[0035] In a preferred embodiment of this application, in step 2, the multi-task convolutional neural network model includes a shared feature extraction layer and two parallel output heads; the first output head is used for binary classification detection of vehicle presence and outputs the sound signal detection confidence; the second output head is used for three-class classification recognition of vehicle type and outputs the vehicle type recognition result, which includes light vehicles, medium vehicles and heavy vehicles.

[0036] In a preferred embodiment of this application, the processing of the sound signal in step 2 includes: converting the sound signal into a spectrogram through short-time Fourier transform, performing frequency occlusion and time occlusion data augmentation processing on the spectrogram using SpecAugment technology, and inputting the augmented spectrogram into the multi-task convolutional neural network model to simultaneously obtain the sound signal detection confidence and the vehicle type recognition result.

[0037] In a preferred embodiment of this application, in step 3, the weather state factor is determined by: acquiring the image brightness and contrast values ​​captured by the camera, as well as the laser return signal intensity of the laser transmitter, comprehensively judging whether the current weather state is sunny, foggy, or rainy, and generating different weather state factors accordingly.

[0038] In a preferred embodiment of this application, in step 4, the formula for calculating the overall detection confidence level is C=α. 声 ×C 声 +α 光 ×C 光 Where Csound is the confidence level of the sound signal detection, Csound 光 To detect the confidence level of the optical device, α 声 and α 光 These are the corresponding weight coefficients and satisfy α. 声 +α 光 =1; when the weather state factor corresponds to a sunny day, α 光 >α 声 When the weather state factor corresponds to foggy or rainy days, α 声 ≥α 光 This is to compensate for the decrease in detection accuracy of optical equipment under adverse weather conditions.

[0039] In a preferred embodiment of this application, in step 4, the differentiated warning specifically includes: when the identified vehicle type is a light vehicle, a standard warning mode is triggered; when the identified vehicle type is a medium vehicle, an enhanced warning mode is triggered, and the warning duration is extended compared to the standard warning mode; when the identified vehicle type is a heavy vehicle, an emergency warning mode is triggered, and the warning duration is further extended compared to the standard warning mode, while the warning signal strength is increased.

[0040] In a preferred embodiment of this application, step 3 further includes: when the optical device detects that a vehicle is driving towards the location of the road to be warned, collecting road image data of the location of the road to be warned at this time; during the time when the optical device detects that no vehicle is driving towards the location of the road to be warned, collecting road image data of the location of the road to be warned at equal time intervals to obtain idle time road image data.

[0041] In a preferred embodiment of this application, processing the off-peak road image data includes: detecting whether the off-peak road image data contains vehicles using a YOLO object detection model; if vehicles are present, detecting whether the multi-task convolutional neural network model in step 2 simultaneously detects vehicles moving towards the location of the road to be warned; if no vehicles are detected in step 2, recording the sound signal transmitted through the road surface at this time, and using the spectrum image corresponding to the sound signal and the vehicle type labeling result output by the YOLO object detection model as a dataset to retrain and optimize the multi-task convolutional neural network model in step 2.

[0042] The locations of roads to be warned include the road ahead of traffic light intersections, sharp turn areas, and road construction areas.

[0043] This application also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program, and the computer program, when executed by the processor, implements a road information warning method.

[0044] Example 1: This example uses the location of the road ahead of a sharp bend on a mountain road as the location to be warned, and provides a detailed description of the road information warning method provided in this application. Mountain roads have limited visibility, making it difficult for vehicles to perceive slow-moving oncoming or same-direction vehicles before a bend, making them a high-risk scenario for traffic accidents. This example installs a detection device 150 meters ahead of the sharp bend, including a sound sensor, a laser emitter, a camera, and an edge computing unit, and installs a three-level controllable warning sign (including adjustable intensity lights and a loudspeaker) at the entrance to the sharp bend.

[0045] Step 1: Acquisition of sound signals;

[0046] The sound sensor is installed at a shallow depth of approximately 5 cm below the road surface to continuously collect vibration sound signals transmitted through the road surface as vehicles travel. The vibration signals generated by vehicle tires on the road surface have significantly lower frequency characteristics compared to ambient noise propagating through the air, and are minimally affected by weather conditions, making them suitable as an auxiliary detection method in adverse weather conditions. The collected raw sound signals are converted from analog to digital and input into subsequent processing stages with a sampling rate of 48 kHz and 512 sampling points per frame.

[0047] Step 2: Multi-task audio signal processing;

[0048] (1) Spectrum transformation: Perform short-time Fourier transform on the acquired sound signal;

[0049] (2) Data Augmentation: The spectrogram is augmented using SpecAugment technology, including: ① Frequency Occlusion: Randomly select a continuous region in the frequency dimension with a width not exceeding 20% ​​of the total frequency range of the spectrum, and set the corresponding pixel value to zero; ② Temporal Occlusion: Randomly select a continuous region in the time dimension with a width not exceeding 10% of the total time length of the spectrum, and set the corresponding pixel value to zero. The purpose of data augmentation is to improve the generalization ability of the multi-task convolutional neural network model to different environmental noises.

[0050] (3) Multi-task convolutional neural network model inference: The multi-task convolutional neural network model consists of a shared feature extraction layer (containing 4 convolutional blocks, each convolutional block containing a convolutional layer, a batch normalization layer, a ReLU activation layer, and a max pooling layer) and two parallel fully connected output heads. The first output head (vehicle presence detection head) outputs a binary classification probability, and the probability corresponding to the vehicle class is taken as the sound signal detection confidence C_sound∈[0,1]; the second output head (vehicle type recognition head) outputs a three-class classification probability, and the category corresponding to the highest probability (light vehicle / medium vehicle / heavy vehicle) is taken as the vehicle type recognition result T.

[0051] In the training phase of this embodiment, a joint loss function is used: L=L 存在 +λ·L 类型 L 存在 For binary classification cross-entropy loss, L 类型 The cross-entropy loss is denoted by 3, and λ=0.8 is the balance coefficient. The training dataset contains 500 spectrogram samples each of light vehicles (cars, small SUVs), medium vehicles (medium buses, medium trucks), and heavy vehicles (large trucks, construction vehicles) collected historically for this road segment.

[0052] 3. Step 3: Optical equipment inspection and weather condition perception;

[0053] (1) Laser emitter detection: A 905nm wavelength laser emitter is installed at the detection location, and the horizontal scan covers the width of the road. When the laser beam is blocked by a vehicle, the receiver detects the change in the intensity of the returned signal, determines that a vehicle has passed, and records the current average intensity I of the returned laser signal. laser .

[0054] (2) Camera detection: The camera acquires road images at 1080P resolution and 15fps frame rate. When a vehicle is detected by the laser emitter, the camera simultaneously acquires road images for subsequent processing by the YOLO target detection model; when no vehicle is detected by the laser emitter, the camera acquires idle road images at a frequency of one frame every 10 seconds.

[0055] (3) Weather condition factor determination: The edge computing unit analyzes the latest frame image from the camera, calculates the global average brightness Ibright and image contrast Icontrast, and combines the laser return signal intensity Ibright with the image brightness Ibright and contrast Ibtrast. laser Determine the weather condition according to the following rules:

[0056] When Ibright≥150 and Icontrast≥40 and I laser ≥0.8×I baseline When the weather is clear, the weather state factor W = 1.

[0057] When Ibright < 150 or Icontrast < 40 and I laser ≥0.5×I baseline At that time, it was determined to be a foggy day, and the weather state factor W=2;

[0058] When I laser <0.5×I baseline At that time, it was determined to be a rainy day, and the weather state factor W=3.

[0059] Among them, I baseline This represents the baseline return signal strength of the laser emitter in clear, unobstructed conditions.

[0060] IV. Step 4: Adaptive weighted fusion decision-making and differentiated early warning;

[0061] (1) Assigning weighting coefficients: Based on the weather state factor W, assign weighting coefficients as follows:

[0062] Sunny day (W=1): α 声 =0.3, α 光 =0.7;

[0063] Foggy day (W=2): α 声 =0.5, α 光 =0.5;

[0064] Rainy day (W=3): α 声 =0.65, α 光 =0.35.

[0065] (2) Calculation of overall confidence level: C=α 声 ×C 声 +α 光 ×C 光 C 光 The normalized detection confidence level of the laser emitter output (0.9 when a vehicle is detected, and 0.1 when no vehicle is detected).

[0066] (3) Warning Judgment and Execution: When C exceeds the preset threshold θ=0.6, it is determined that a vehicle is driving towards the location of the road to be warned, and a differentiated warning is executed based on the vehicle type identification result T:

[0067] Light vehicles (T=light): Trigger standard warning mode, activate warning sign lights (brightness 50%), duration 10 seconds;

[0068] Medium-sized vehicles (T=medium): Trigger enhanced warning mode, activate warning sign lights (brightness 80%) and intermittent audible sound for 20 seconds;

[0069] Heavy vehicles (T=heavy): Trigger emergency warning mode, activate warning sign lights (100% brightness) and sound a continuous audible alarm for 35 seconds.

[0070] The design basis for the above warning duration is as follows: the braking distance of light vehicles is about 40 to 60 meters (at a speed of 60 km / h), medium vehicles are about 60 to 80 meters, and heavy vehicles are about 80 to 120 meters. Based on the distance of the detection position from the sharp turn entrance to 150 meters, the corresponding warning advance amount is set for each type of vehicle.

[0071] V. Online optimization mechanism;

[0072] During system operation, when C 光 The corresponding laser emitter detected a vehicle, but the overall detection confidence C did not exceed the threshold θ (i.e., the confidence C of the sound model judgment). 声 When there are deficiencies, the system still performs differentiated warnings (at this time, the single-channel detection result of the laser emitter is used as the standard), and at the same time records the sound signal transmitted on the road surface. The corresponding spectrum image, together with the vehicle type labeling result identified by the laser emitter / camera, is used as a new training sample to update the multi-task convolutional neural network model in order to improve its judgment accuracy in this scenario.

[0073] During off-peak hours when the laser emitter does not detect vehicles, a road image is captured every 10 seconds. The YOLO object detection model (using a pre-trained YOLOv8 model) is then used to detect whether vehicles are present in the off-peak images. If YOLO detects a vehicle but the multi-task convolutional neural network model does not (C... 声 If the value is less than 0.5, the road surface sound signal at that moment is recorded, and its spectrum image and the vehicle category label output by YOLO are used as new training samples for batch retraining of the multi-task convolutional neural network model. After two weeks of field operation, the sound detection accuracy of the multi-task convolutional neural network model for vehicles on this road section can be improved from the initial 82% to over 91%.

Claims

1. A road information early warning method, characterized in that, The method includes the following steps: Step 1: Collect the sound signal transmitted through the road surface at the location of the road to be warned; Step 2: Process the sound signal, detect whether a vehicle is driving towards the location of the warning road using a multi-task convolutional neural network model, and at the same time identify the vehicle type of the sound signal to determine whether the driving vehicle is a light vehicle, medium vehicle or heavy vehicle. Step 3: Use optical equipment to detect whether any vehicles are driving towards the location of the road to be warned, and determine the weather condition factor based on the current weather conditions; Step 4: Based on the weather state factor, assign corresponding weight coefficients to the sound signal detection confidence level in Step 2 and the optical device detection confidence level in Step 3, and calculate the comprehensive detection confidence level; when the comprehensive detection confidence level exceeds a preset threshold, determine that a vehicle is driving towards the location of the road to be warned, and execute differentiated warnings based on the vehicle type identified in Step 2. Step 4 further includes: when the optical device in step 3 detects a vehicle moving toward the location of the road to be warned but the comprehensive detection confidence does not exceed the preset threshold, a differentiated warning is still executed, and the sound signal transmitted through the road surface at this time is recorded and used as training data to optimize the multi-task convolutional neural network model in step 2 online.

2. The road information early warning method according to claim 1, characterized in that: In step 2, the multi-task convolutional neural network model includes a shared feature extraction layer and two parallel output heads; the first output head is used for binary classification detection of vehicle presence and outputs the sound signal detection confidence; the second output head is used for three-class classification recognition of vehicle type and outputs the vehicle type recognition result, which includes light vehicles, medium vehicles and heavy vehicles.

3. The road information early warning method according to claim 2, characterized in that: The processing of the sound signal in step 2 includes: converting the sound signal into a spectrogram through short-time Fourier transform; performing frequency occlusion and time occlusion data augmentation processing on the spectrogram using SpecAugment technology; and inputting the augmented spectrogram into the multi-task convolutional neural network model to obtain the sound signal detection confidence and the vehicle type recognition result.

4. The road information early warning method according to claim 1, characterized in that: In step 3, the weather state factor is determined by: acquiring the image brightness and contrast values ​​collected by the camera, as well as the laser return signal intensity of the laser transmitter, comprehensively judging whether the current weather state is sunny, foggy, or rainy, and generating different weather state factors accordingly.

5. The road information early warning method according to claim 1, characterized in that: In step 4, the formula for calculating the overall detection confidence level is C=α. 声 ×C 声 +α 光 ×C 光 Where Csound is the confidence level of the sound signal detection, Csound 光 To detect the confidence level of the optical device, α 声 and α 光 These are the corresponding weight coefficients and satisfy α. 声 +α 光 =1; when the weather state factor corresponds to a sunny day, α 光 >α 声 When the weather state factor corresponds to foggy or rainy days, α 声 ≥α 光 This is to compensate for the decrease in detection accuracy of optical equipment under adverse weather conditions.

6. The road information early warning method according to claim 1, characterized in that: In step 4, the differentiated warning specifically means: when the identified vehicle type is a light vehicle, a standard warning mode is triggered; when the identified vehicle type is a medium vehicle, an enhanced warning mode is triggered, and the warning duration is extended compared to the standard warning mode; when the identified vehicle type is a heavy vehicle, an emergency warning mode is triggered, and the warning duration is further extended compared to the standard warning mode, while the warning signal strength is increased.

7. The road information early warning method according to claim 1, characterized in that: Step 3 further includes: when the optical device detects that a vehicle is driving towards the location of the road to be warned, road image data of the location of the road to be warned is collected at this time; during the time when the optical device detects that no vehicle is driving towards the location of the road to be warned, road image data of the location of the road to be warned is collected at equal time intervals to obtain idle time road image data.

8. A road information early warning method according to claim 7, characterized in that: Processing the off-peak road image data includes: detecting whether the off-peak road image data contains vehicles using a YOLO object detection model; if vehicles are present, detecting whether the multi-task convolutional neural network model in step 2 simultaneously detects vehicles moving towards the location of the road to be warned; if no vehicles are detected in step 2, recording the sound signal transmitted through the road surface at this time, and using the spectrum image corresponding to the sound signal and the vehicle type labeling result output by the YOLO object detection model as a dataset to retrain and optimize the multi-task convolutional neural network model in step 2.

9. A road information early warning method according to claim 1, characterized in that: The locations of roads to be warned include the road ahead of traffic light intersections, sharp turn areas, and road construction areas.

10. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements a road information warning method as described in any one of claims 1 to 9.

Images