Control method and device for priority passing of fire rescue vehicle at intersection

Abstract

The invention discloses an intersection priority passing control method and device for a fire rescue vehicle, and the method comprises the steps: S1, carrying out the multi-modal data collection of vehicles at an intersection; S2, carrying out the judgment of a fire fighting truck: fusing the multi-modal data collected in S1, verifying the features after fusion, judging the vehicle as the fire fighting truck if the verification is passed, and entering S3; if the verification is not passed, returning to S1; and S3, according to the data acquired in the S1 and the determination result in the S2, calculating the time for the fire fighting truck to arrive at the intersection, and then adjusting the traffic light time. And through multi-modal feature fusion and an intelligent decision-making mechanism, the priority passing control efficiency of the fire rescue vehicle at the intersection is remarkably improved. Visual features, namely, the reflectivity of a red coating special for a fire fighting truck, structured recognition of an emergency license plate, voiceprint features, namely time-frequency analysis of an alarm whistle, and a three-dimensional data source of a vehicle motion state, namely real-time speed, are integrated, and a high-robustness fire fighting vehicle judgment system is constructed.

Description

Technical Field

[0001] The present invention relates to a method and device for controlling the priority passage of fire rescue vehicles at an intersection, belonging to the technical field of traffic passage. Background Art

[0002] In modern urban life, firefighting and rescue operations shoulder the crucial mission of protecting people's lives and property, placing particular emphasis on speed of response and efficient traffic flow. However, when firefighting and rescue vehicles are performing emergency tasks and need to quickly cross urban intersections, they often face the challenges of complex and ever-changing traffic environments.

[0003] Currently, the mainstream methods for providing right-of-way to emergency vehicles rely primarily on pre-set priority control programs in traffic lights or on-site control by traffic police. However, these traditional approaches have significant limitations in practical application. Traffic light priority control schemes are typically based on fixed, pre-set rules, making them difficult to dynamically and flexibly adjust to real-time traffic flow conditions and lacking adaptability. Traffic police on-site control is constrained by limited human resources, personal field of view, and complex intersection environments. This makes efficient and comprehensive traffic control difficult, especially at large or complex intersections. Although recent developments in intelligent transportation systems have attempted to provide better access for emergency vehicles, the reliability and accuracy of commonly used recognition technologies, such as traditional license plate recognition, are severely affected by factors such as changing ambient lighting conditions and partial occlusion of the target vehicle. Existing systems generally lack the ability to effectively identify and utilize the distinctive features of fire and rescue vehicles, such as their specific vehicle paint colors, high-frequency flashing warning lights, and distinctive siren sounds. These systems fail to fully utilize these distinctive features to significantly improve recognition efficiency and robustness. Summary of the Invention

[0004] Purpose of the invention: In response to the deficiencies in the prior art, the present invention provides a method and device for controlling the priority passage of fire rescue vehicles at intersections. The present invention significantly improves the efficiency of the priority passage control of fire rescue vehicles at intersections through multimodal feature fusion and intelligent decision-making mechanism.

[0005] Technical solution: A method for controlling the priority passage of fire trucks at an intersection, comprising the following steps:

[0006] S1. Collect multimodal data from vehicles at the intersection:

[0007] S101. Obtain vehicle images at the intersection and input them into a YOLOv5s model trained with a standard dataset to identify the emergency license plate of the fire truck.

[0008] S102, based on the vehicle image at the intersection, extracting the red paint features of the fire truck at a wavelength of 610-650nm, and calculating the reflectivity of the red paint of the fire truck;

[0009] S103, collecting environmental sound wave signals and obtaining siren time-frequency characteristics to determine whether the fire truck is on duty;

[0010] S104, collecting the real-time speed V of the vehicle;

[0011] S2, fire truck determination: The multimodal data collected in S1 is fused and the fused features are verified. If the verification passes, it is determined to be a fire truck and enters S3; if the verification fails, it returns to S1;

[0012] S3. Calculate the time it takes for the fire truck to arrive at the intersection based on the data collected by S1 and the determination result of S2, and then adjust the traffic light time.

[0013] Preferably, the S101 is specifically:

[0014] The YOLOv5s model training is specifically as follows:

[0015] The fire truck emergency license plate is defined as an independent detection category Emergency_Plate. The standard dataset includes a set of standard emergency license plates. All emergency license plates are marked with a unified category box. The complete license plate area is selected. The complete license plate area includes the province abbreviation, the issuing authority code, the serial number, and the word "Emergency".

[0016] The YOLOv5s model automatically learns key features through convolutional neural networks:

[0017] S1011. First, establish a spatial position constraint mechanism and define the end area of the license plate, that is, the width Δw and height Δh of the end of the license plate. Calculate the sum of the activation intensity of the "emergency" feature in this area as the key factor of the category probability. The sum of the activation intensity of the "emergency" feature φ emeergency The (x,y) formula is:

[0018]

[0019] Where: σ is the sigmoid activation function; W c energency is the convolution kernel weight for the word “emergency”; F c (x,y) is the eigenvalue of the cth channel at position (x,y); b is the bias term;

[0020] The probability P of the emergency license plate category determined by the activation intensity of the terminal area class for:

[0021]

[0022] Among them: 1 layout (x, y) is a spatial position indicating function, which is 1 when the coordinates (x, y) are in the end region and 0 otherwise; f(z) represents a probability mapping function Its core function is to convert the weighted sum of the activation intensity in the end region into the final class probability P class ; x = w - Δw represents the starting abscissa of the spatial position, w represents the total width of the detected license plate area, and △w represents the width of the end region of the license plate; y = h - △h represents the starting ordinate of the spatial position, h represents the total height of the detected license plate area, and △h is the height of the end region of the license plate;

[0023] S1012. The YOLOv5s model learns the stroke structures of the characters "应" and "急" and verifies that they conform to the topological relationship of the standard font library:

[0024] Extract stroke key points from the vehicle image, construct a stroke topology graph, and calculate the similarity score between the detected stroke turning angle and the standard angle, the stroke structure similarity score S topology The calculation is as follows:

[0025]

[0026] Among them, N is the number of strokes of the character "应"; θ k is the turning angle of the a-th stroke detected by the model; is the theoretical angle of the a-th stroke specified by the GA36 standard; τ θ is the angle tolerance threshold;

[0027] S1013. The YOLOv5s model verifies the contrast between the character and the background through the red-white boundary color difference ΔE value in the Lab color space, and calculates the average color difference C between the red character pixels and the adjacent white background pixels within the character area △E , and converts this color difference into a probability value p color :

[0028]

[0029] Among them: (L x , a x , b x ) and (L y , a y , b y ) are the Lab values of the red character pixels and the white background pixels respectively; Ω represents the set of image coordinates;

[0030] The average color difference C △E and the class probability pcolor The correlation function is:

[0031]

[0032] Where: u is the color difference threshold; k is the sensitivity coefficient;

[0033] S1014, comprehensive spatial layout, stroke topology and color difference verification three factors to calculate the final emergency license plate category probability p final :

[0034] p final =p class ·s topology ·p color

[0035] When the fire truck’s overall characteristic target is detected, the actual vehicle image at the intersection is input into the YOLOv5s model to obtain the fire truck emergency license plate recognition result p final .

[0036] Preferably, the S102 is specifically:

[0037] The specific calculation of the reflectivity of the red paint on the fire truck is:

[0038]

[0039] Where λ represents the wavelength of the light wave; I(λ) represents the light intensity signal at the wavelength λ; S color Red paint reflectivity for fire trucks.

[0040] Preferably, the S103 is specifically:

[0041] Collect the acoustic wave amplitude s(t) at the sampling point at the intersection at time sequence number t; perform pre-emphasis processing on s(t) in sequence:

[0042] S pre (t) = s(t) - αs(t-1)

[0043] Where: α represents the pre-emphasis coefficient (high-frequency enhancement strength) α∈[0.9,0.99], S pre (t) is the pre-emphasis signal;

[0044] Then, the frame length is 25ms, the frame shift is 10ms, and the Hamming window is used for frame segmentation and windowing. A Wiener filter based on power spectrum is used to retain the main frequency band of the siren of 300–3500Hz for noise suppression.

[0045] Perform fast Fourier transform FFT on each frame signal in turn:

[0046] Calculate the power spectrum: P(f) = |FFT(sframe (t))| 2 ; Among them, s frame (t) is s frame (t) = s pre (t)·Q(tt m ), t∈[t m ,t m +L-1], t m represents the starting time index of the mth frame, t represents the global discrete time index, and Q represents the Hamming window function L represents the frame length, that is, the number of sampling points contained in a single frame;

[0047] Through 26 Mel-scale triangular filter banks H m (f) Filtering in the frequency range 0–4 kHz to calculate the Mel energy M m :

[0048] M m =∑P(f)·H m (f)

[0049] Perform logarithmic compression and DCT cosine transform processing:

[0050] Where n = 0, 1, ..., 12

[0051] Obtain 13-dimensional static MFCC coefficients, and then calculate the first-order difference coefficients:

[0052] (△):△C n (t) = C n (t+1)-C n (t-1)

[0053] Among them, △C n (t) is a dynamic feature, which indicates the rate of change of MFCC coefficients in three adjacent frames; C n (t+1) represents the MFCC coefficient calculated on the t+1 frame signal; C n (t-1) represents the MFCC coefficient calculated on the t-1 frame signal;

[0054] Finally, a 26-dimensional MFCC voiceprint feature vector is generated: x = [C1,...,C 13 ,△C1...,△C 13 ] T ;

[0055] Substitute the 26-dimensional MFCC voiceprint feature vector x into the Gaussian matching formula to obtain the matching probability of the fire truck being on duty:

[0056]

[0057] Where: N(x|u i ,∑i) represents the matching probability between the current voiceprint feature x and the i-th Gaussian component; x represents the input 26-dimensional MFCC voiceprint feature vector; u i represents the mean vector of the i-th Gaussian component of the preset fire truck alarm sound feature library; ∑i represents the covariance matrix of the i-th Gaussian component; k1 represents the feature dimension, where k1 = 26; T represents the transpose operation.

[0058] Preferably, the S2 is specifically:

[0059] Primary verification is performed using the fire truck's red paint feature and the fire truck's emergency license plate recognition results:

[0060] The composite scoring formula for primary verification based on the reflectivity of the fire truck's red paint and the emergency license plate is:

[0061] Confidence v =0.6·S color +0.4·p final

[0062] p final =p class ·s topology ·p color

[0063]

[0064] Where λ represents the wavelength of the light wave, I(λ) represents the light intensity signal at the wavelength λ, and S color The reflectivity of the red paint used for fire trucks is p final The final emergency license plate category probability is obtained based on YOLOv5 recognition, ranging from 0 to 1.

[0065] When the comprehensive score Confidence v When the value is ≥0.8, the advanced verification is carried out after the primary verification. The advanced verification is as follows:

[0066] Cross-validation is performed by fusing the data obtained from S101 and S102 with the data obtained from S103:

[0067]

[0068] Normalization processing:

[0069] Among them, Norm(j) is the normalization of the data; j represents Confidence v and N(x|u i ,∑i) calculated value;jmin Indicates the minimum effective value of the system. The system sets the minimum effective threshold to 0.3; j max The theoretical maximum value is 1.0;

[0070] If Score m >0.8, the target is confirmed to be a fire truck.

[0071] Preferably, the time for the fire truck to arrive at the intersection is calculated in S3 as follows:

[0072] According to the real-time vehicle speed V collected in S104, assume that the fire truck is currently at data collection point A, the current lane end point is C, and the remaining time of the green light is G remain , the remaining time of the red light is R remain , the green light extension time is △G, and the red light reduction time is △R. Then the expected arrival time ETA of the fire truck from the current moment to the lane end point C is expressed as: Where D is the distance from A to C.

[0073] Preferably, the traffic light time adjustment process in S3 is specifically as follows:

[0074] If the fire truck's turning type is straight ahead and the current traffic light is green, ETA≥G remain ,but:

[0075] △G=max(3,ETA-G remain )

[0076] Traffic light adjustment strategy is to extend the green light by △G seconds;

[0077] If the fire truck's turn type is straight ahead and the current traffic light is red, then:

[0078] △R=max(0,R remain -3)

[0079] Traffic light adjustment strategy is to reduce the red light time by △R seconds and set a 3-second yellow light safety time for the opposite lane;

[0080] If the fire truck's turning type is left turn and the current vehicle's traffic light is green, ETA≥G remain ,but:

[0081] △G=max(3,ETA-G remain )

[0082] Traffic light adjustment strategy is to extend the green light by △G seconds;

[0083] If the fire truck's turn type is left turn and the current vehicle traffic light is red, then:

[0084] △R=max(0,R remain -3)

[0085] Traffic light adjustment strategy is to reduce the red light time by △R seconds and set a 3-second yellow light safety time for the opposite lane;

[0086] If the fire truck turns right, the traffic light adjustment strategy needs to consider pedestrian safety. If the pedestrian indicator traffic light is green, all lights will be set to yellow for 3 seconds and then turn red. If the light is red, the red light time will be extended. The extended time is the time it takes for the fire truck to arrive at the intersection.

[0087] A device for implementing a method for controlling fire trucks' priority passage at an intersection, comprising a visual recognition module, a sound recognition module, an intelligent algorithm module, and a signal light control module, all of which are arranged at a data collection point A;

[0088] The visual recognition module is used to capture the fire red features of fire trucks at intersections, vehicle license plates and real-time speed parameters in real time;

[0089] The sound recognition module is used to capture the sound emitted by the vehicle, extract the sound features using a sound processing algorithm, and compare it with a preset fire truck alarm sound feature library to further identify the status of the fire truck;

[0090] The intelligent algorithm module calculates the time when the vehicle arrives at and passes through the intersection based on the data, and calculates the expected phase state of the traffic light at the above time, that is, at the above time, the traffic light is red, yellow or green;

[0091] The signal light control module controls the traffic light to output a passable phase state at the above time according to the time when the vehicle arrives at and passes through the intersection generated by the intelligent algorithm module.

[0092] Beneficial effects: The present invention significantly improves the efficiency of priority traffic control for fire rescue vehicles at intersections through multimodal feature fusion and intelligent decision-making mechanism. By integrating visual features, namely the reflectivity of the fire truck's special red paint, the structured recognition of the emergency license plate, the voiceprint features, namely the time-frequency analysis of the siren, and the vehicle's motion status, namely the real-time speed, a three-dimensional data source, a highly robust fire truck identification system is constructed. This system breaks through the limitations of traditional license plate recognition technology that is susceptible to environmental interference. It enhances the recognition reliability of emergency license plates through spatial layout constraints, stroke topology verification and color difference detection. At the same time, it uses the unique spectral reflectance characteristics and alarm voiceprint characteristics of fire trucks to form a dual physical identification verification, which significantly enhances the anti-interference ability and accuracy of target recognition in complex traffic environments. Based on the cross-validation mechanism of multimodal data fusion, dynamic high-confidence confirmation of the identity of the fire truck is achieved, effectively avoiding the risk of false triggering and missed detection. By calculating the precise arrival time of vehicles at intersections in real time and matching traffic phase conditions, a dynamic traffic light control strategy is generated that balances efficiency and safety. This strategy adaptively adjusts green light durations or red light durations based on different driving directions (straight ahead, left turns), as well as green / red light phase differences, and provides a yellow light safety buffer in the opposing lane. Furthermore, for right turns, pedestrian signal conditions are synchronized to prioritize pedestrian safety. This method systematically optimizes the safety and coordination of the intersection's overall traffic flow while ensuring the rapid passage of rescue vehicles, providing intelligent technical support for urban emergency rescue systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0094] Figure 1 is a flow chart of the method of the present invention;

[0095] Figure 2 Schematic diagram of the intersection and data collection point A of the present invention. DETAILED DESCRIPTION

[0096] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0097] In the description of the present invention, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0098] In the present invention, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0099] like Figure 1 As shown, a method for controlling priority passage of fire trucks at an intersection includes the following steps:

[0100] S1. Collect multimodal data from vehicles at the intersection:

[0101] S101. Obtain vehicle images at the intersection and input them into a YOLOv5s model trained with a standard dataset to identify the emergency license plate of the fire truck.

[0102] The YOLOv5s model training is specifically as follows:

[0103] The fire truck emergency license plate is defined as an independent detection category Emergency_Plate. The standard dataset includes a set of standard emergency license plates. All emergency license plates are marked with a unified category box. The complete license plate area is selected. The complete license plate area includes the province abbreviation, the issuing authority code, the serial number, and the word "Emergency".

[0104] The YOLOv5s model automatically learns key features through convolutional neural networks:

[0105] S1011. First, establish a spatial position constraint mechanism and define the end area of the license plate, that is, the width Δw and height Δh of the end of the license plate. Calculate the sum of the activation intensity of the "emergency" feature in this area as the key factor of the category probability. The sum of the activation intensity of the "emergency" feature φ emeergency The (x,y) formula is:

[0106]

[0107] Where: σ is the sigmoid activation function; W c energency is the convolutional kernel weight for the word "emergency"; F c (x, y) is the eigenvalue of the c-th channel at the position (x, y); b is the bias term;

[0108] The probability P of the emergency license plate category determined by the activation intensity of the end region class is:

[0109]

[0110] Where: 1 layout (x, y) is the spatial position indicator function, which is 1 when the coordinates (x, y) are in the end region and 0 otherwise; f(z) represents the probability mapping function Its core function is to convert the weighted sum of the activation intensity of the end region into the final category probability P class ; x = w - Δw represents the starting abscissa of the spatial position, w represents the total width of the detected license plate area, and △w represents the width of the end region of the license plate; y = h - △h represents the starting ordinate of the spatial position, h represents the total height of the detected license plate area, and △h is the height of the end region of the license plate;

[0111] S1012. The YOLOv5s model learns the stroke structures of the characters "ying" and "ji" and verifies that they conform to the topological relationship of the standard font library:

[0112] Extract the stroke key points from the vehicle image, construct a stroke topology graph, and calculate the similarity score between the detected stroke turning angle and the standard angle. The stroke structure similarity score S topology is calculated as follows:

[0113]

[0114] Where, N is the number of strokes of the character "ying"; θ k is the turning angle of the a-th stroke detected by the model; is the theoretical angle of the a-th stroke specified by the GA36 standard; τ θ is the angle tolerance threshold;

[0115] S1013. The YOLOv5s model verifies the contrast between the character and the background through the red-white boundary color difference ΔE value in the Lab color space, and calculates the average color difference C between the red character pixels and the adjacent white background pixels within the character region △E , and converts this color difference into a probability value p color :

[0116]

[0117] Among them: (L x ,a x ,b x ) and (L y ,a y ,b y ) are the Lab values of red character pixels and white background pixels respectively; Ω represents the image coordinate set;

[0118] Average color difference C △E and class probability p color The correlation function is:

[0119]

[0120] Where: u is the color difference threshold; k is the sensitivity coefficient;

[0121] S1014, comprehensive spatial layout, stroke topology and color difference verification three factors to calculate the final emergency license plate category probability p final :

[0122] p final =p class ·s topology ·p color

[0123] When the fire truck’s overall characteristic target is detected, the actual vehicle image at the intersection is input into the YOLOv5s model to obtain the fire truck emergency license plate recognition result p final .

[0124] S102, based on the vehicle image at the intersection, extracting the red paint features of the fire truck at a wavelength of 610-650nm, and calculating the reflectivity of the red paint of the fire truck;

[0125] The specific calculation of the reflectivity of the red paint on the fire truck is:

[0126]

[0127] Where λ represents the wavelength of the light wave; I(λ) represents the light intensity signal at the wavelength λ; S color Red paint reflectivity for fire trucks.

[0128] S103, collecting environmental sound wave signals and obtaining siren time-frequency characteristics to determine whether the fire truck is on duty;

[0129] Collect the acoustic wave amplitude s(t) at the sampling point at the intersection at time sequence number t; perform pre-emphasis processing on s(t) in sequence:

[0130] S pre(t) = s(t) - αs(t-1)

[0131] Where: α represents the pre-emphasis coefficient (high-frequency enhancement strength) α∈[0.9,0.99], S pre (t) is the pre-emphasis signal;

[0132] Then, the frame length is 25ms, the frame shift is 10ms, and the Hamming window is used for frame segmentation and windowing. A Wiener filter based on power spectrum is used to retain the main frequency band of the siren of 300–3500Hz for noise suppression.

[0133] Perform fast Fourier transform FFT on each frame signal in turn:

[0134] Calculate the power spectrum: P(f) = |FFT(s frame (t))| 2 ; Among them, s frame (t) is s frame (t) = s pre (t)·Q(tt m ), t∈[t m ,t m +L-1], t m represents the starting time index of the mth frame, t represents the global discrete time index, and Q represents the Hamming window function L represents the frame length, that is, the number of sampling points contained in a single frame;

[0135] Through 26 Mel-scale triangular filter banks H m (f) Filtering in the frequency range 0–4 kHz to calculate the Mel energy M m :

[0136] M m =∑P(f)·H m (f)

[0137] Perform logarithmic compression and DCT cosine transform processing:

[0138] Where n = 0, 1, ..., 12

[0139] Obtain 13-dimensional static MFCC coefficients, and then calculate the first-order difference coefficients:

[0140] (△):△C n (t) = C n (t+1)-C n (t-1)

[0141] Among them, △C n (t) is a dynamic feature, which indicates the rate of change of MFCC coefficients in three adjacent frames; C n(t+1) represents the MFCC coefficient calculated on the t+1 frame signal; C n (t-1) represents the MFCC coefficient calculated on the t-1 frame signal;

[0142] Finally, a 26-dimensional MFCC voiceprint feature vector is generated: x = [C1,...,C 13 ,△C1...,△C 13 ] T ;

[0143] Substitute the 26-dimensional MFCC voiceprint feature vector x into the Gaussian matching formula to obtain the matching probability of the fire truck being on duty:

[0144]

[0145] Where: N(x|u i ,∑i) represents the matching probability between the current voiceprint feature x and the i-th Gaussian component; x represents the input 26-dimensional MFCC voiceprint feature vector; u i represents the mean vector of the i-th Gaussian component of the preset fire truck alarm sound feature library; ∑i represents the covariance matrix of the i-th Gaussian component; k1 represents the feature dimension, where k1 = 26; T represents the transpose operation.

[0146] S104, collecting the real-time speed V of the vehicle;

[0147] S2, fire truck determination: The multimodal data collected in S1 is fused and the fused features are verified. If the verification passes, it is determined to be a fire truck and enters S3; if the verification fails, it returns to S1;

[0148] Primary verification is performed using the fire truck's red paint feature and the fire truck's emergency license plate recognition results:

[0149] The composite scoring formula for primary verification based on the reflectivity of the fire truck's red paint and the emergency license plate is:

[0150] Confidence v =0.6·S color +0.4·p final

[0151] p final =p class ·s topology ·p color

[0152]

[0153] Where λ represents the wavelength of the light wave, I(λ) represents the light intensity signal at the wavelength λ, and S colorThe reflectivity of the red paint used for fire trucks is p final The final emergency license plate category probability is obtained based on YOLOv5 recognition, ranging from 0 to 1.

[0154] When the comprehensive score Confidence v When the value is ≥0.8, the advanced verification is carried out after the primary verification. The advanced verification is as follows:

[0155] Cross-validation is performed by fusing the data obtained from S101 and S102 with the data obtained from S103:

[0156]

[0157]

[0158] Normalization processing:

[0159] Among them, Norm(j) is the normalization of the data; j represents Confidence v and N(x|u i ,∑i) calculated value;j min Indicates the minimum effective value of the system. The system sets the minimum effective threshold to 0.3; j max The theoretical maximum value is 1.0;

[0160] If Score m >0.8, the target is confirmed to be a fire truck.

[0161] Example 1

[0162] The fire truck approaches A at 60 km / h (16.67 m / s), collects the vehicle body reflection spectrum from 400-700 nm, calculates the energy proportion of the 610-650 nm band to be 0.844, and the YOLOv5s model detects the license plate category probability as 0.97. v =0.6×0.844+0.4×0.97=0.8944,Confidence v >0.8 Pass primary verification and trigger advanced verification; collect sound wave signals, extract 26-dimensional MFCC features, compare the current MFCC features with the pre-stored fire truck voiceprint library, and calculate the matching probability score N(x|u i ,∑i) is 0.95, normalized to 0.9, primary verification visual confidence is 0.8944, normalized to 0.849, comprehensive score Score m =0.6×0.849+0.4×0.9=0.8694, Score m >0.8, it is determined to be an on-duty fire truck.

[0163] S3. Calculate the time it takes for the fire truck to arrive at the intersection based on the data collected by S1 and the determination result of S2, and then adjust the traffic light time.

[0164] The time it takes for a fire truck to arrive at an intersection is calculated as follows:

[0165] like Figure 2 As shown, according to the real-time vehicle speed V collected in S104, the fire truck is currently at data collection point A, the current lane end point is C, and the remaining time of the green light is G remain , the remaining time of the red light is R remain , the green light extension time is △G, and the red light reduction time is △R. Then the expected arrival time ETA of the fire truck from the current moment to the lane end point C is expressed as: Where D is the distance from A to C.

[0166] The traffic light time adjustment process is as follows:

[0167] If the fire truck's turning type is straight ahead and the current traffic light is green, ETA≥G remain ,but:

[0168] △G=max(3,ETA-G remain )

[0169] Traffic light adjustment strategy is to extend the green light by △G seconds;

[0170] If the fire truck's turn type is straight ahead and the current traffic light is red, then:

[0171] △R=max(0,R remain -3)

[0172] Traffic light adjustment strategy is to reduce the red light time by △R seconds and set a 3-second yellow light safety time for the opposite lane;

[0173] If the fire truck's turning type is left turn and the current vehicle's traffic light is green, ETA≥G remain ,but:

[0174] △G=max(3,ETA-G remain )

[0175] Traffic light adjustment strategy is to extend the green light by △G seconds;

[0176] If the fire truck's turn type is left turn and the current vehicle traffic light is red, then:

[0177] △R=max(0,R remain -3)

[0178] Traffic light adjustment strategy is to reduce the red light time by △R seconds and set a 3-second yellow light safety time for the opposite lane;

[0179] If the fire truck turns right, the traffic light adjustment strategy needs to consider pedestrian safety. If the pedestrian indicator traffic light is green, all lights will be set to yellow for 3 seconds and then turn red. If the light is red, the red light time will be extended. The extended time is the time it takes for the fire truck to arrive at the intersection.

[0180] Example 2

[0181] The fire rescue vehicle approaches A at 60 km / h (16.67 m / s). The intelligent algorithm module receives the fire rescue vehicle information and knows that the fire truck will arrive at the intersection in about 6 seconds. If the straight traffic light shows green for 3 seconds and the left turn traffic light shows red for 15 seconds, the straight green light will be extended by 6-3=3 seconds, and the left turn red light will be adjusted to green after 3 seconds. The opposite lane will immediately trigger a yellow light for 3 seconds and then enter the red light cycle. If the pedestrian traffic light is green, it will all be set to yellow for 3 seconds and then turn red. If it is red, the red light time will be extended. The extended time is the time it takes for the fire truck to arrive at the intersection.

[0182] Example 3

[0183] A fire rescue vehicle approaches intersection A at 60 km / h (16.67 m / s). The intelligent algorithm module receives information about the fire rescue vehicle and predicts that the fire truck will arrive at the intersection in approximately 6 seconds. If the straight-ahead traffic light is green for 8 seconds and the left-turn traffic light is green for 15 seconds, the left-turn and straight-ahead traffic lights will not be adjusted. To ensure pedestrian safety, the pedestrian traffic light is set to yellow for 3 seconds before turning red. If the light is red, the red light is extended by the time it takes for the fire truck to arrive at the intersection.

[0184] A device for implementing a method for controlling fire trucks' priority passage at an intersection, characterized by comprising a visual recognition module, a sound recognition module, an intelligent algorithm module, and a signal light control module, all of which are arranged at a data collection point A;

[0185] The visual recognition module is used to capture the fire red features of fire trucks at intersections, vehicle license plates and real-time speed parameters in real time;

[0186] The sound recognition module is used to capture the sound emitted by the vehicle, extract the sound features using a sound processing algorithm, and compare it with a preset fire truck alarm sound feature library to further identify the status of the fire truck;

[0187] The intelligent algorithm module calculates the time when the vehicle arrives at and passes through the intersection based on the data, and calculates the expected phase state of the traffic light at the above time, that is, at the above time, the traffic light is red, yellow or green;

[0188] The signal light control module controls the traffic light to output a passable phase state at the above time according to the time when the vehicle arrives at and passes through the intersection generated by the intelligent algorithm module.

[0189] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0190] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for controlling priority passage of fire trucks at an intersection, characterized by: It includes the following steps: S1. Collect multi-modal data of vehicles at the intersection: S101. Obtain the vehicle images at the intersection, input them into the YOLOv5s model trained with a standard dataset, and identify the emergency license plates of fire trucks; S102. Based on the vehicle images at the intersection, extract the red paint features of fire trucks with wavelengths of 610 - 650 nm, and calculate the reflectivity of the red paint of fire trucks; S103. Collect environmental acoustic signals, obtain the time-frequency features of sirens, and determine whether the fire truck is on duty; S104. Collect the real-time vehicle speed V; S2. Fire truck determination: Integrate the multi-modal data collected in S1, and verify the integrated features. If the verification passes, it is determined as a fire truck and proceed to S3; if the verification fails, return to S1; S3. According to the data collected in S1 and the determination result of S2, calculate the time for the fire truck to reach the intersection, and then adjust the traffic light time.

2. The fire truck intersection priority control method according to claim 1, characterized in that: The specific content of S101 is as follows: The specific training of the YOLOv5s model is as follows: Define the emergency license plate of the fire truck as an independent detection category Emergency_Plate. The standard dataset includes a set of standard emergency license plates. All emergency license plates are labeled with a unified category box, and the complete license plate area is framed. The complete license plate area includes the abbreviation of the province, the code of the license-issuing authority, the serial number, and the word "Emergency"; The steps for the YOLOv5s model to automatically learn key features through a convolutional neural network are as follows: S1011. First, establish a spatial position constraint mechanism and define the end area of the license plate, that is, the width Δw and height Δh of the end of the license plate. Calculate the sum of the activation intensity of the "emergency" feature in this area as the key factor of the category probability. The sum of the activation intensity of the "emergency" feature φ emeergency The (x,y) formula is: Where: σ is the sigmoid activation function; W c energency is the convolution kernel weight for the word "emergency"; F c (x,y) is the eigenvalue of the cth channel at position (x,y); b is the bias term; The probability P of the emergency license plate category determined by the activation intensity of the terminal area class for: Of which: 1 layout (x, y) is the spatial position indicator function, which is 1 when the coordinate (x, y) is in the end area, otherwise it is 0; f(z) represents the probability mapping function Its core function is to convert the weighted sum of the activation strength of the terminal area into the final category probability P class ; x = w-Δw represents the starting point of the abscissa of the spatial position, w represents the total width of the detected license plate area, and Δw represents the width of the end area of the license plate; y =h-△h represents the vertical coordinate starting point of the spatial position, h represents the total height of the detected license plate area, and △h is the height of the license plate end area; S1012. The YOLOv5s model learns the stroke structures of the characters "Ying" and "Ji" and verifies that they conform to the topological relationship of the standard font library; Extract stroke key points from vehicle images, construct stroke topology, calculate the similarity score between the detected stroke turning angle and the standard angle, and the stroke structure similarity score S topology The calculation is as follows: where N is the number of strokes of the character "应"; θ k is the turning angle of the a-th stroke detected by the model; is the theoretical angle of the a-th stroke specified by the GA36 standard; τ θ is the angle tolerance threshold; The S1013 and YOLOv5s models verify the contrast between characters and backgrounds by using the red-white boundary color difference ΔE value in the Lab color space. The average color difference C between red character pixels and adjacent white background pixels is calculated within the character area. △E , and convert the color difference into a probability value p color : Among them: (L x ,a x ,b x ) and (L y ,a y ,b y ) are the Lab values of red character pixels and white background pixels respectively; Ω represents the image coordinate set; Average color difference C △E and class probability p color The correlation function is: Where: u is the color difference threshold; k is the sensitivity coefficient; S1014, comprehensive spatial layout, stroke topology and color difference verification three factors to calculate the final emergency license plate category probability p final : p final =p class ·s topology ·p color When the fire truck’s overall characteristic target is detected, the actual vehicle image at the intersection is input into the YOLOv5s model to obtain the fire truck emergency license plate recognition result p final .

3. The fire truck intersection priority control method according to claim 2, characterized in that: The specific content of S102 is as follows: The calculation of the reflectivity of the red paint of fire trucks is specifically as follows: Where λ represents the wavelength of the light wave; I(λ) represents the light intensity signal at the wavelength λ; S color Red paint reflectivity for fire trucks.

4. The fire truck intersection priority control method according to claim 3 is characterized in that: The specific content of S103 is as follows: Collect the acoustic amplitude s(t) of the sampling point at the t-th time sequence at the intersection; perform pre-emphasis processing on s(t) in sequence: S pre (t)=s(t)-αs(t-1) Where: α represents the pre-emphasis coefficient (high-frequency enhancement strength) α∈[0.9,0.99], S pre (t) is the pre-emphasis signal; Then, segment it with a frame length of 25 ms and a frame shift of 10 ms, and perform frame windowing with Hamming window smoothing; use Wiener filtering based on the power spectrum to retain the main frequency band of the siren at 300 - 3500 Hz for noise suppression; Perform fast Fourier transform FFT on each frame of the signal in sequence: Calculate the power spectrum: P(f) = |FFT(s frame (t))| 2 ; Among them, s frame (t) is s frame (t) = s pre (t)·Q(tt m ), t∈[t m ,t m +L-1], t m represents the starting time index of the mth frame, t represents the global discrete time index, and Q represents the Hamming window function L represents the frame length, that is, the number of sampling points contained in a single frame; Through 26 Mel-scale triangular filter banks H m (f) Filtering in the frequency range 0–4 kHz to calculate the Mel energy M m : M m =∑P(f)·H m (f) Perform logarithmic compression and DCT cosine transform processing: Where n = 0, 1, ..., 12 Obtain 13-dimensional static MFCC coefficients, and then append and calculate the first-order difference coefficients: (△):△C n (t)=C n (t+1)-C n (t-1) Among them, △C n (t) is a dynamic feature, which indicates the rate of change of MFCC coefficients in three adjacent frames; C n (t+1) represents the MFCC coefficient calculated on the t+1 frame signal; C n (t-1) represents the MFCC coefficient calculated on the t-1 frame signal; Finally, a 26-dimensional MFCC voiceprint feature vector is generated: x = [C1,...,C 13 ,△C1...,△C 13 ] T ; Finally, substitute the generated 26-dimensional MFCC voiceprint feature vector x into the Gaussian matching formula to obtain the matching probability that the fire truck is on duty; Where: N(x|u i ,∑i) represents the matching probability between the current voiceprint feature x and the i-th Gaussian component; x represents the input 26-dimensional MFCC voiceprint feature vector; u i represents the mean vector of the i-th Gaussian component of the preset fire truck alarm sound feature library; ∑i represents the covariance matrix of the i-th Gaussian component; k1 represents the feature dimension, where k1 = 26; T represents the transpose operation.

5. The fire truck intersection priority control method according to claim 4 is characterized in that: The specific content of S2 is as follows: Perform primary verification through the red paint features of fire trucks and the recognition results of emergency license plates of fire trucks: The composite scoring formula for primary verification through the reflectivity of the special red paint of fire trucks and emergency license plates is: Confidence v =0.6·S color +0.4·p final p final =p class ·s topology ·p color Where λ represents the wavelength of the light wave, I(λ) represents the light intensity signal at the wavelength λ, and S color The reflectivity of the red paint used for fire trucks is p final The final emergency license plate category probability is obtained based on YOLOv5 recognition, ranging from 0 to 1. When the comprehensive score Confidence v When the value is ≥0.8, the advanced verification is carried out after the primary verification. The advanced verification is as follows: Perform cross-verification by integrating the data obtained from S101, S102 and the data obtained from S103; Normalization processing: Among them, Norm(j) is the normalization of the data; j represents Confidence v and N(x|u i ,∑i) calculated value;j min Indicates the minimum effective value of the system. The system sets the minimum effective threshold to 0.3; j max The theoretical maximum value is 1.0; If Score m >0.8, the target is confirmed to be a fire truck.

6. The fire truck intersection priority control method according to claim 1, characterized in that: The specific calculation of the time for the fire truck to reach the intersection in S3 is as follows: According to the real-time vehicle speed V collected in S104, assume that the fire truck is currently at data collection point A, the current lane end point is C, and the remaining time of the green light is G remain , the remaining time of the red light is R remain , the green light extension time is △G, and the red light reduction time is △R. Then the expected arrival time ETA of the fire truck from the current moment to the lane end point C is expressed as: Where D is the distance from A to C.

7. The fire truck intersection priority control method according to claim 6, characterized in that: The specific process of adjusting the traffic light time in S3 is as follows: If the fire truck's turning type is straight ahead and the current traffic light is green, ETA≥G remain ,but: △G=max(3,ETA-G remain ) The traffic light adjustment strategy is to extend the green light by △G seconds; If the turning type of the fire truck is straight and the current vehicle indicates that the traffic light is red, then: △R=max(0,R remain -3) Traffic light adjustment strategy is to reduce the red light time by △R seconds and set a 3-second yellow light safety time for the opposite lane; If the fire truck's turning type is left turn and the current vehicle's traffic light is green, ETA≥G remain ,but: △G=max(3, ETA-G remain ) Traffic light adjustment strategy is to extend the green light by △G seconds; If the fire truck's turn type is left turn and the current vehicle traffic light is red, then: △R=max(0,R remain -3) Traffic light adjustment strategy is to reduce the red light time by △R seconds and set a 3-second yellow light safety time for the opposite lane; If the fire truck turns right, the traffic light adjustment strategy needs to consider pedestrian safety. If the pedestrian indicator traffic light is green, all lights will be set to yellow for 3 seconds and then turn red. If the light is red, the red light time will be extended. The extended time is the time it takes for the fire truck to arrive at the intersection.

8. A device for implementing the fire truck intersection priority control method according to any one of claims 1 to 7, characterized in that: It includes a visual recognition module, a voice recognition module, an intelligent algorithm module, and a signal light control module, all of which are arranged at the data collection point A; The visual recognition module is used to capture the fire red features of fire trucks at intersections, vehicle license plates and real-time speed parameters in real time; The sound recognition module is used to capture the sound emitted by the vehicle, extract the sound features using a sound processing algorithm, and compare it with a preset fire truck alarm sound feature library to further identify the status of the fire truck; The intelligent algorithm module calculates the time when the vehicle arrives at and passes through the intersection based on the data, and calculates the expected phase state of the traffic light at the above time, that is, at the above time, the traffic light is red, yellow or green; The signal light control module controls the traffic light to output a passable phase state at the above time according to the time when the vehicle arrives at and passes through the intersection generated by the intelligent algorithm module.

Images