Traffic horn capture method and device based on acoustic localization and image coordination

Abstract

This invention discloses a method and device for capturing traffic horn honking based on acoustic localization and image coordination, belonging to the field of signal processing technology. It includes visual detection and tracking: real-time detection and tracking of vehicles within a monitored area to generate a list of visual targets; the list includes at least one suspect vehicle and its spatiotemporal coordinate sequence motion trajectory; acoustic verification: based on the visual target list, a microphone array is controlled to perform acoustic localization of the suspect vehicle's location, obtaining sound source location information; comprehensive confidence calculation: spatial matching and verification of the sound source location information and the suspect vehicle's motion trajectory are performed to calculate the confidence of the honking event, determine the occurrence of the honking event, and initially lock onto the target vehicle. This invention transforms wide-area acoustic detection into directional verification of specific visual targets through directional beamforming and multi-dimensional acoustic feature recognition, effectively suppressing environmental noise and interference, and solving the problems of multiple sound source overlap and large localization errors at intersections.

Description

Technical Field

[0001] This invention relates to the field of signal processing technology, specifically to a method and device for capturing traffic horn sounds based on acoustic positioning and image coordination. Background Technology

[0002] Traffic noise is one of the main sources of urban environmental noise. Sudden car horns can startle people, causing increased heart rate and blood pressure. Long-term exposure can damage hearing and increase the risk of cardiovascular disease. Continuous horn noise can cause irritability, anxiety, and difficulty concentrating, seriously affecting quality of life and mental health. Currently, significant progress has been made in capturing illegal horn use, but conventional technical methods still have shortcomings: environmental noise interference, parallel traffic, and dense traffic can lead to false alarms; and they cannot distinguish between emergency horn use and malicious horn use.

[0003] For example, patent publication number CN109448389A discloses an intelligent detection method for car horns. This method uses an air sonar sensor array to receive sound signals, amplifies, filters, and samples the received signals; classifies car horn signals and environmental noise using a trained support vector machine; estimates the direction of the sound source signal by changing the signal incident angle θ through mechanical scanning; locates the violating vehicle using beamforming positioning; and finally captures evidence of the violation using a high-definition camera. This method can monitor sound signals on the road in real time, accurately identify horn sounds, and locate vehicles in real time based on the collected horn sounds.

[0004] However, the above and similar technical solutions still have the following shortcomings: the lack of an effective multi-sound source separation mechanism leads to mutual interference of positioning signals when multiple vehicles honk their horns at the same time, resulting in a decrease in positioning accuracy; the lack of a strong correlation method between acoustic and visual evidence leads to the risk of spatiotemporal asynchrony between the sound source positioning results and the images captured by the camera, resulting in insufficient credibility of the evidence chain. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for capturing traffic horn sounds based on acoustic positioning and image coordination, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a traffic horn-honking capture method based on acoustic localization and image coordination, comprising:

[0007] Visual detection and tracking: Real-time detection and tracking of vehicles within the monitored area, generating a list of visual targets; the list of visual targets includes the motion trajectory of at least one suspect vehicle and its spatiotemporal coordinate sequence;

[0008] Acoustic verification: Based on the visual target list, the microphone array is controlled to locate the location of the suspect vehicle by sound waves and obtain the location information of the sound source;

[0009] Comprehensive confidence calculation: Spatial matching and verification of the sound source location information with the movement trajectory of the suspect vehicle is performed to calculate the confidence of the horn honking event, determine the occurrence of the horn honking event, and initially locate the target vehicle;

[0010] Dynamic evidence association: When a horn honking event is determined, the horn audio data and its timestamp are recorded, and the camera equipment is controlled in parallel to collect image information of the area where the target vehicle is located, and micro-vibration analysis is performed on the image information;

[0011] Illegal horn detection and evidence generation: Based on the contextual intelligent analysis model, the necessity of horn horn blasting events is determined according to audio data, micro-vibration analysis results and horn horn blasting necessity rules, and evidence of illegal horn horn blasting is generated.

[0012] Furthermore, the visual detection and tracking steps include:

[0013] The video stream of the monitored area is decoded and preprocessed in real time to identify the initial bounding boxes of all vehicles in the picture.

[0014] Each detected vehicle is assigned a unique identifier. Based on the tracking algorithm, the position of each vehicle in consecutive video frames is predicted and updated. The pixel coordinates in the image sequence are converted in real time into a spatiotemporal coordinate sequence that is consistent with the microphone array, and a motion trajectory is formed.

[0015] The detected vehicles are subjected to semantic analysis of attributes and dynamic behavior to identify their interaction states with the surrounding environment; the interaction states include normal following, accelerating to approach the vehicle in front, forced lane change, and the presence of an obstacle to cut in front.

[0016] Based on the active area of ​​potential sound sources calculated in real time using the microphone array, and combined with the semantics of vehicle dynamic behavior, vehicles that show potential horn-honking motives are marked as suspect vehicles, and a list of visual targets is output.

[0017] Furthermore, the method for obtaining the sound source location information includes:

[0018] Receive and parse the list of visual targets, and control the microphone array to dynamically generate at least one directional detection beam, focusing on the direction of the suspect vehicle;

[0019] The original acoustic signals in each detection beam channel are separated, enhanced, and feature-analyzed to identify acoustic signals that match the characteristics of a whistle.

[0020] For acoustic signals that pass the initial screening, calculate their sound source location coordinates and associate them with the vehicle identifier that triggered the calculation to generate sound source location information guided by visual targets.

[0021] Furthermore, the method for acquiring the sound source location information includes correction of the original acoustic signal under adverse weather conditions:

[0022] Real-time acquisition of meteorological data for the monitored area, including wind speed, wind direction, temperature, humidity, and rainfall;

[0023] Based on real-time meteorological data, the speed of sound in the current environment is dynamically calculated, and a sound wave propagation model including the influence of wind speed vector is established to correct the basic physical parameters in the sound wave localization algorithm.

[0024] Based on the sound wave propagation model, a wind noise model is constructed, and wind noise filtering is applied to the original acoustic signal of the microphone array at the front end of acoustic signal processing.

[0025] Furthermore, the sound wave propagation model dynamically and continuously calculates the time difference of sound waves propagating from any point in space to each microphone based on the received real-time meteorological data; the wind noise model predicts the statistical characteristics of the noise generated by the wind on the microphones and its spatial correlation in the microphone array based on the received real-time wind speed and wind direction data and combined with the physical configuration of the microphone array, and generates a wind noise reference signal.

[0026] Furthermore, the steps for calculating the overall confidence level include:

[0027] Map the sound source's location coordinates and the suspect vehicle's trajectory to the same spatiotemporal reference frame;

[0028] Based on the trajectory of the suspect vehicle, the spatial overlap between it and the location of the sound source in a continuous time series is calculated, and the spatiotemporal variation characteristics of the overlapping area are analyzed.

[0029] The motion pattern of the sound source location information is extracted and the motion consistency analysis is performed with the motion trajectory of the suspect vehicle to obtain the motion consistency analysis results.

[0030] By combining the dynamic behavioral semantics and acoustic signal features of the suspect vehicle, a joint visual-audio vector is constructed.

[0031] Based on the fusion results of spatial overlap, motion consistency analysis, and audio-visual joint vectors, the overall confidence level of the horn-honking event is calculated using a weighted average.

[0032] The overall confidence level is compared with the adaptive threshold to determine the occurrence of the horn honking event and initially identify the target vehicle.

[0033] Furthermore, the adaptive threshold is automatically adjusted based on weather conditions and vehicle density.

[0034] Furthermore, the dynamic evidence association step includes:

[0035] When a horn-honking event is determined to have occurred, the original acoustic signal that triggered the determination is stored; the camera equipment is controlled in parallel to acquire image sequences of key parts of the target vehicle; the key parts include the front grille and the hood;

[0036] The stored raw acoustic signal is preprocessed to extract its core spectral features;

[0037] Based on the phase motion amplification algorithm, micro-vibration analysis is performed on the acquired image sequence to obtain the vibration response waveform of key parts of the target vehicle under acoustic wave impact, and signal analysis is performed to extract the modal features of vibration.

[0038] By comparing the core spectral characteristics with the modal characteristics of vibration, the consistency of acoustic and vibration characteristics is verified, and the final target vehicle is located.

[0039] Furthermore, the method for constructing the contextual intelligence analysis model includes:

[0040] Based on the video stream of the monitored area, the traffic situation in the monitored area is quantified in real time at the moment of horn blasting; the traffic situation includes the smoothness of the road ahead, the density of nearby vehicles, and the conflict distance between pedestrians and non-motorized vehicles and the target vehicle.

[0041] The core spectral features, vibration modal features, motion state vectors of the final target vehicle, vehicle interaction status, and traffic situation are spatiotemporally aligned and feature fused.

[0042] Based on the fused information, the driving scenario is reconstructed and matched with the horn necessity rule to determine whether there is an emergency situation that allows horn use, and the scenario matching result is obtained.

[0043] Based on the context matching results and the acoustic characteristics of the horn blast, a final determination of illegality is made, and evidence of illegal horn blasting is generated.

[0044] The traffic horn capture device based on acoustic localization and image coordination is characterized by using the traffic horn capture method based on acoustic localization and image coordination described above.

[0045] Compared with the prior art, the beneficial effects of the present invention are:

[0046] The traffic horn capture method and device based on acoustic localization and image coordination, through traffic situation quantification and driving context reconstruction, places horn-honking behavior in a specific traffic environment for necessity rule matching, thereby distinguishing necessary horn-honking behaviors such as emergency avoidance and avoiding misjudgment of reasonable horn-honking behavior.

[0047] Meanwhile, by using directional beamforming and multidimensional acoustic feature recognition, wide-area acoustic detection is transformed into directional verification of specific visual targets, effectively suppressing environmental noise and interference, and solving the problems of multiple sound source overlap and large positioning errors at intersections. By establishing a unified spatiotemporal benchmark and synchronously linking acoustic and vibration evidence, precise alignment of acoustic events and visual trajectories and micro-vibration analysis are achieved, reducing the risk of equipment asynchrony and ensuring a strong correlation between horn-honking behavior and target vehicles in the evidence chain. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the traffic horn capture method of the present invention;

[0049] Figure 2 This is a schematic diagram of the visual detection and tracking method of the present invention;

[0050] Figure 3 This is a schematic diagram illustrating the effect of the visual detection and tracking method of the present invention at a crossroads.

[0051] Figure 4 This is a schematic diagram of the sound source location information acquisition method of the present invention;

[0052] Figure 5 This is a schematic diagram illustrating the acoustic signal separation and feature recognition effect of the present invention;

[0053] Figure 6 This is a schematic diagram of the comprehensive confidence level calculation method of the present invention;

[0054] Figure 7 This is a schematic diagram of the dynamic evidence association method of the present invention;

[0055] Figure 8 This is a schematic diagram illustrating the effect of acoustic-vibration correlation determination in this invention;

[0056] Figure 9 This is a schematic diagram of the contextual intelligent analysis model of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, the present invention provides a technical solution: a method and apparatus for capturing traffic horn sounds based on acoustic positioning and image coordination, comprising:

[0059] Step 1: Visual Detection and Tracking. Vehicles within the monitored area are detected and tracked in real time, generating a list of visual targets. This list includes the motion trajectory of at least one suspect vehicle and its spatiotemporal coordinate sequence.

[0060] like Figure 2 As shown, the present invention provides a visual detection and tracking method;

[0061] Specifically:

[0062] The video stream of the monitored area is decoded and preprocessed in real time to identify the initial bounding boxes of all vehicles in the picture.

[0063] Each detected vehicle is assigned a unique identifier. Based on the tracking algorithm, the position of each vehicle in consecutive video frames is predicted and updated. The pixel coordinates in the image sequence are converted in real time into a spatiotemporal coordinate sequence that is consistent with the microphone array, and a motion trajectory is formed.

[0064] Perform semantic analysis on the attributes and dynamic behavior of the detected vehicles to identify their interaction with the surrounding environment;

[0065] Based on the active area of ​​potential sound sources calculated in real time using the microphone array, and combined with the semantics of vehicle dynamic behavior, vehicles that show potential horn-honking motives are marked as suspect vehicles, and a list of visual targets is output.

[0066] It is important to note the vehicle detection and initialization process. The monitoring video stream undergoes real-time decoding and image preprocessing, including image denoising, contrast enhancement, and size normalization. Subsequently, a deep learning-based object detection model (such as YOLO or Faster R-CNN series models) is used to identify all vehicles in the image and output their initial bounding boxes. Optionally, non-maximum suppression algorithms and confidence threshold filtering are employed to eliminate redundant detection boxes and output accurate initial vehicle bounding boxes.

[0067] Multi-target tracking and trajectory generation. To assign and maintain unique identifiers for continuously detected vehicles, a multi-target tracking algorithm (such as the Hungarian algorithm combining motion models and appearance features) is used to achieve stable tracking across video frames. For each successfully tracked vehicle, the following steps are performed to generate a trajectory: coordinate transformation, converting its position in each frame (e.g., the center point of the bounding box bottom edge) to world coordinates in real time through camera calibration and perspective transformation; sequence construction, binding each newly transformed world coordinate with its corresponding high-precision timestamp and appending it chronologically to the data structure corresponding to the vehicle identifier to construct its spatiotemporal coordinate sequence; state estimation, applying a filtering algorithm (such as Kalman filtering) to smooth the spatiotemporal coordinate sequence to generate the vehicle's motion trajectory. Then, based on the smoothed trajectory data, the instantaneous motion state vector of the vehicle, including velocity, acceleration, and heading angle, is estimated using methods such as numerical differentiation.

[0068] Vehicle dynamic behavior semantic analysis. Based on spatiotemporal coordinate sequences and instantaneous motion state vectors, vehicle attribute recognition and dynamic behavior semantic understanding are performed: Vehicle attributes are identified by using convolutional neural networks to analyze vehicle bounding box images and identify static attributes such as vehicle type and color; Environmental context awareness is achieved by using image segmentation models to identify lane lines and combining target detection results to track the positions and trajectories of other traffic participants (vehicles and pedestrians) to obtain real-time information on lane layout and other traffic participants; Dynamic behavior semantic understanding integrates the vehicle's spatiotemporal coordinate sequences, instantaneous motion state vectors, and environmental context, and uses predefined behavioral reasoning logic to label the vehicle's interaction states with behavioral semantic tags, including "forced lane change," "acceleration," etc. The judgment rules for "approaching the vehicle in front", "obstacle to cutting in front", and "normal following" include: when the vehicle's lateral acceleration exceeds the threshold, its trajectory briefly crosses the lane line, and the turn signal is not activated, it is judged as "forced lane change"; when the vehicle's longitudinal acceleration is positive, and the time distance to the vehicle in front in the same lane continues to decrease and falls below the safety threshold, it is judged as "accelerating to approach the vehicle in front"; when other road users cut in front of the vehicle, causing the time distance to the vehicle to shorten rapidly and fall below the safety threshold, and the vehicle brakes (negative acceleration), it is judged as "obstacle to cutting in front"; when the vehicle is stably within the lane line, the absolute value of the acceleration is below the threshold, and the time distance to the vehicle in front in the same lane is greater than the safety threshold (e.g., 2.0 seconds), it is judged as "normal following".

[0069] Visual target list generation. Based on acoustic signals collected by the microphone array, a sound source localization algorithm is used for preliminary sound source region detection to calculate one or more approximate potential sound source active areas within the monitored area. Combined with the dynamic behavior semantics of the vehicles, a suspicion assessment is performed on each tracked vehicle, and a comprehensive suspicion score S is calculated, where S=W. b ×F b +W a×F a +W t ×F t , where: F b This is a behavioral semantic score. Based on a predefined behavioral semantic-score mapping table, different behavioral semantics are assigned corresponding scores, such as 0.9 for forced lane change, 0.7 for accelerating to approach the vehicle in front, 0.5 for the presence of an obstacle cutting in front, and 0 for normal following; F a The acoustic region fraction is calculated by taking the shortest distance *d* from the vehicle's current position (taken from the latest coordinates in its spatiotemporal coordinate sequence) to the boundary of the acoustically active region, and then calculating *F* using the distance-fraction function. a If the vehicle is within the area (d≤0), then F a =1, the vehicle is outside the area but within a certain buffer distance (0<d≤D). 缓冲 ), then F a =1.0-(d / D) 缓冲 Vehicles are far from the area (d > D) 缓冲 ), then F a =0;F t It is the trajectory quality score, indirectly evaluated based on the spatiotemporal coordinate sequence, F t =(F 平滑 +F 长度 ) / 2, where: F 平滑 This is the smoothness score, calculated based on the instantaneous acceleration of each small segment of the trajectory using continuous coordinate points. The statistical value (such as the standard deviation) of the acceleration for the entire sequence is then calculated. The smaller the statistical value, the more stable the acceleration, and the higher the trajectory smoothness score. F 长度 It is a length fraction, calculated based on the length of the spatiotemporal coordinate sequence (i.e., the number of coordinate points it contains). The longer the sequence, the higher the length fraction; W b W a W t They are F b F a F t The corresponding weight coefficients, and W b +W a +W t =1, these weights can be obtained through expert experience presets or machine learning model optimization.

[0070] A preset comprehensive suspicion score threshold is established. The scores of all vehicles are compared to this threshold, and vehicles with scores exceeding the threshold are officially marked as "suspect vehicles." All relevant information for each suspect vehicle is packaged into a complete data entry, and all entries together constitute a visual target list. This visual target list is a dynamically updated data set in memory, typically existing as an array or list. Each entry (corresponding to a suspect vehicle) includes the following fields: vehicle identifier, vehicle static attributes, current world coordinates, spatiotemporal coordinate sequence, behavioral semantic label, and comprehensive suspicion score. For example, ... Figure 3 As shown, the black outline represents the initial bounding box of the detected vehicles within the monitoring area. "ID258", "ID259", "ID260", and "ID261" are unique identifiers assigned to the vehicles. The black lines represent the vehicle's trajectory, and the black dashed lines represent the predicted trajectory. The phrases "normal following" are behavioral semantic labels. The gray shading around ID258 represents the active area of ​​a potential sound source. A comprehensive suspicion score is calculated for each of the four vehicles: W... b =0.5, W a =0.3、W t =0.2, D=30m, and the comprehensive suspicion score threshold is 0.6; S 258 =0.5×0.9+0.3×1.0+0.2×0.88=0.926, S 259 =0.5×0+0.3×0.9+0.2×0.78=0.426, S 260 =0.5×0.7+0.3×0.8+0.2×0.85=0.76, S 261 =0.5×0.5+0.3×0.7+0.2×0.83=0.626, that is, S 258 >S 260 >S 261 >0.6>S 259 Therefore, the vehicles with identifiers ID258, ID260, and ID261 are suspected vehicles.

[0071] Step 2: Acoustic Verification. Based on the visual target list, control the microphone array to perform sound wave localization of the suspect vehicle's location and obtain sound source location information.

[0072] like Figure 4 As shown, the present invention provides a method for obtaining sound source location information;

[0073] Specifically:

[0074] Receive and parse the list of visual targets, and control the microphone array to dynamically generate at least one directional detection beam, focusing on the direction of the suspect vehicle;

[0075] The original acoustic signals in each detection beam channel are separated, enhanced, and feature-analyzed to identify acoustic signals that match the characteristics of a whistle.

[0076] For acoustic signals that pass the initial screening, calculate their sound source location coordinates and associate them with the vehicle identifier that triggered the calculation to generate sound source location information guided by visual targets.

[0077] It is important to note the directional beamforming and spatial focusing involved. The visual target list is received and parsed to obtain the latest world coordinates for each suspect vehicle. Based on these coordinates and the fixed position of the microphone array, the azimuth and pitch angles of the vehicle relative to the array are calculated. Subsequently, digital beamforming technology (such as a minimum variance distortionless response beamformer) is used to control the microphone array to dynamically generate at least one directional detection beam, whose main lobe direction is precisely aligned with the location of the suspect vehicle, achieving acoustic spatial focusing to enhance the signal from the target direction and suppress interference.

[0078] Acoustic signal separation and feature recognition. The raw acoustic signals output from each detection beam channel are preprocessed, including anti-aliasing filtering and analog-to-digital conversion (if the front end is an analog microphone array). The mixed multi-channel acoustic signals are separated by a beamformer (e.g., an adaptive beamformer) to obtain independent acoustic signal streams corresponding to different suspected vehicle directions. The separated channel acoustic signals are then enhanced (e.g., using adaptive filtering and spectral subtraction algorithms) to improve the signal-to-noise ratio. Subsequently, feature analysis is performed on the enhanced acoustic signals to extract acoustic features characterizing vehicle horn sounds, such as sound pressure level, fundamental frequency, harmonic structure, duration, and Mel-frequency cepstral coefficients. The extracted features are input into a pre-trained classifier (e.g., a support vector machine) for recognition, outputting the confidence level that the signal belongs to a horn sound. When the recognition confidence level is higher than a preset threshold, the channel signal is determined to be an acoustic signal matching the horn characteristics and passes the initial screening; otherwise, the signal is discarded. For example, a recognition confidence threshold of 0.8 is used. Figure 5 This is a schematic diagram illustrating the effect of sound source signal separation and feature recognition in this embodiment. It is a polar coordinate graph including recognition confidence and sound source orientation. The radius (0, 0.2, 0.4, 0.6, 0.8, 1.0) represents the recognition confidence (the further out, the more like a horn). The angle (0°, 30°, 60°, 90°) represents the beam direction (sound source orientation), 0° is directly in front of the vehicle, and 90° is to the right of the vehicle. The hollow circle represents the initial screening result. The larger the hollow circle, the greater the probability of passing the initial screening. As shown in the figure, the recognition confidence of the 0° direction is 0.92, which matches the acoustic signal of a horn and passes the initial screening. The recognition confidence of the 30° direction is 0.88, which also passes the initial screening. Signals from other directions are discarded.

[0079] Sound source location calculation and information association. For acoustic signals that pass the initial screening, a high-resolution sound source localization algorithm (such as the generalized cross-correlation function method) is used to calculate their precise sound source location coordinates. The calculated sound source location coordinates are then bound to the identifier of the suspect vehicle that triggered the acoustic detection, establishing an association record of "vehicle identifier - sound source location coordinates - timestamp," ultimately generating visual target-oriented sound source location information.

[0080] To improve positioning accuracy in adverse weather conditions, an acoustic signal correction step is also included, specifically:

[0081] Real-time meteorological data acquisition. A meteorological sensor network is deployed within or near the monitoring area to collect environmental meteorological parameters in real time; the meteorological sensors include, but are not limited to, anemometers, wind vanes, temperature and humidity sensors, and rain gauges. The collected meteorological data is transmitted in real time to the acoustic signal processing unit via wired / wireless communication interfaces.

[0082] Sound velocity correction and sound wave propagation model construction. Based on real-time collected temperature and humidity data, the sound velocity value under the current environment is accurately calculated using an empirical formula and updated to the sound source localization algorithm. The empirical formula for sound velocity can be c≈331.3+0.606T, where: c is the speed of sound in m / s; T is the temperature in °C; c is further corrected according to the humidity H, c*≈c+0.06H. Simultaneously, a sound wave localization model that comprehensively considers the influence of wind speed vectors is established to correct the propagation path and speed of sound waves in the wind field. Specific methods include: decomposing the wind speed vector into components parallel to the sound source-microphone connection and perpendicular components, where the parallel component effectively changes the apparent propagation speed of the sound wave. Through this model, the actual path and time delay of sound waves propagating from any point in space to each unit in the microphone array can be dynamically predicted, thereby correcting the localization error caused by the wind field.

[0083] Wind noise suppression. Based on real-time wind speed, wind direction, and the physical configuration of the microphone array, a wind noise model is constructed to predict the statistical characteristics (such as power spectral density) and spatial correlation (i.e., the relationship between wind noise signals received by different microphones), thereby generating a wind noise reference signal. This model utilizes the high spatial correlation of wind noise within the array and the well-defined directionality of the target acoustic signal for signal separation. At the front end of the acoustic signal processing chain (usually before beamforming), wind noise suppression techniques (such as adaptive filters and spatial filtering algorithms) are used to filter the raw acoustic signal of the microphone array to suppress wind noise interference.

[0084] Step 3: Calculate the overall confidence level. Spatially match and verify the sound source location information with the movement trajectory of the suspect vehicle, calculate the confidence level of the horn honking event, determine the occurrence of the horn honking event, and initially locate the target vehicle.

[0085] like Figure 6 As shown, the present invention provides a method for calculating comprehensive confidence level;

[0086] Specifically:

[0087] Map the sound source's location coordinates and the suspect vehicle's trajectory to the same spatiotemporal reference frame;

[0088] Based on the trajectory of the suspect vehicle, the spatial overlap between it and the location of the sound source in a continuous time series is calculated, and the spatiotemporal variation characteristics of the overlapping area are analyzed.

[0089] The motion pattern of the sound source location information is extracted and the motion consistency analysis is performed with the motion trajectory of the suspect vehicle to obtain the motion consistency analysis results.

[0090] By combining the dynamic behavioral semantics and acoustic signal features of the suspect vehicle, a joint visual-audio vector is constructed.

[0091] Based on the fusion results of spatial overlap, motion consistency analysis, and audio-visual joint vectors, the overall confidence level of the horn-honking event is calculated using a weighted average.

[0092] The overall confidence level is compared with the adaptive threshold to determine the occurrence of the horn honking event and initially identify the target vehicle.

[0093] It is important to note the importance of a spatiotemporal reference system mapping. A unified world coordinate system is established as the reference benchmark for all spatial data. The origin and axes of this world coordinate system are determined through joint calibration during deployment to ensure that the spatial relationship between the camera equipment and the microphone array is known and fixed. The azimuth coordinates of the sound source from step two and the motion trajectory of the suspect vehicle from step one are directly placed into this unified world coordinate system. All data points are ensured to have high-precision timestamps, and the acoustic events are precisely aligned with the visual trajectories in time based on these timestamps, establishing a unified spatiotemporal benchmark for subsequent analysis.

[0094] Spatial overlap and spatiotemporal characteristic analysis. Within a sliding time window, the Euclidean distance between the suspect vehicle's location and the sound source's azimuth coordinates is calculated. Spatial overlap is calculated by defining an effective overlap radius and statistically analyzing the proportion of time points where the spatial distance between the vehicle and the sound source is less than this radius, serving as the basic overlap index O. 基础 Overlap stability analysis shows that a longer continuous overlap time indicates a stable positional relationship between the sound source and the vehicle; therefore, the duration of the continuous overlap period is calculated as a stability index. 稳定 Relative motion analysis analyzes the duration of consecutive overlapping time periods and the changing trends of relative distances to assess the stability of spatial relationships.

[0095] Motion consistency analysis. The motion pattern refers to estimating the spatial trajectory and characteristics of a sound source from a series of continuous, short-lived sound source location points, and then comparing it with the visually observed vehicle trajectory to verify synchronization. Specific methods include: constructing a short-time sound source trajectory based on continuous sound source azimuth coordinates through filtering (such as Kalman filtering) or interpolation; then calculating motion consistency metrics, including directional consistency. Within the same time period, the directional changes of the suspect vehicle's trajectory (heading angle sequence) and the sound source's trajectory are calculated separately, and the correlation coefficient between the two directional sequences is calculated as the directional consistency index C. 方向 Speed ​​consistency is assessed by calculating the instantaneous velocity sequences of the suspect vehicle's trajectory and the sound source's trajectory (obtained through numerical differentiation) within the same time period, and then calculating the correlation coefficient between the two velocity sequences as the speed consistency index C. 速度 The similarity of the trajectory shapes is measured by calculating the dynamic time warping distance between the sound source trajectory and the suspect vehicle trajectory during the overlapping time period, given that there are enough points on the trajectory. The smaller the distance, the more consistent the two trajectories are.

[0096] Vision-audio joint vector construction. Visual features and acoustic features are combined to form a vision-audio joint vector; the visual features include scores obtained by quantizing dynamic behavior semantic labels (e.g., 0.9 points for forced lane change, 0.7 points for accelerating to approach the vehicle in front, 0.5 points for the presence of an obstacle cutting in front, and 0.1 points for normal following) and vehicle type weights (e.g., police cars and trucks can be assigned higher prior weights); the acoustic features include acoustic recognition confidence, sound pressure level, signal purity, fundamental frequency stability, and number of harmonics.

[0097] Overall confidence level calculation. A weighted fusion method can be used to calculate the final overall confidence level: normalize the indicators calculated in the second and third steps to the [0, 1] interval; Overall confidence level = W 空间 ×F 空间 +W 运动 ×F 运动 +W 联合 ×F 联合 , where: F 空间 It is a comprehensive score of spatial overlap based on spatial overlap and stability; F 运动 It is a comprehensive score based on the consistency of motion in terms of direction and velocity; F 联合 It is the joint vector score based on key features in the audio-visual joint vector (such as behavioral semantic score and acoustic recognition confidence); W 空间 W 运动 W 联合 These are the corresponding weight coefficients, and their sum is 1. The weight values ​​can be set using historical training data or expert experience.

[0098] Adaptive threshold determination and target locking. The calculated comprehensive confidence score is compared with the adaptive threshold. The adaptive threshold is dynamically adjusted according to the real-time environment, and its calculation formula is: Adaptive threshold = Basic threshold × α × β, where: the basic threshold is set according to expert experience; α is a weather factor based on real-time rainfall and wind speed, α = 1.0 + k1 × rainfall + k2 × wind speed; β is a density factor based on real-time vehicle density in the monitoring area, β = 1.0 + k3 × vehicle density; the coefficients k1, k2, and k3 are determined by conducting experiments under different weather conditions (such as different rainfall and wind speeds) and different traffic densities to analyze the impact of environmental factors on the system detection performance, and are calibrated and determined based on the experimental results. If the overall confidence level is greater than or equal to the adaptive threshold, a horn-honking event is determined to have occurred. This event is then strongly correlated with the currently analyzed suspect vehicle identifier, initially identifying the vehicle as the target. A horn-honking event record is generated, containing the following information: horn-honking event identifier, timestamp, identifier of the identified vehicle, sound source coordinates, overall confidence level, and associated evidence. This associated evidence refers to: visual evidence, linking the vehicle's dynamic behavior semantic tags, its spatiotemporal coordinate sequence, and spatial overlap and motion consistency analysis results; and acoustic evidence, linking the characteristics (such as sound pressure level) of the acoustic signal that triggered the initial screening and its identification confidence level. If the overall confidence level is less than the adaptive threshold, it is marked as a low-confidence horn-honking event for subsequent verification.

[0099] Step 4: Dynamic Evidence Association. When a horn honking event is determined to have occurred, the horn audio data and its timestamp are recorded. In parallel, the camera equipment is controlled to collect image information of the area where the target vehicle is located, and micro-vibration analysis is performed on the image information.

[0100] like Figure 7 As shown, the present invention provides a method for dynamic evidence association;

[0101] Specifically:

[0102] When a horn-honking event is determined to have occurred, the original acoustic signal that triggered the determination is stored; and the camera equipment is controlled in parallel to acquire image sequences of key parts of the target vehicle.

[0103] The stored raw acoustic signal is preprocessed to extract its core spectral features;

[0104] Based on the phase motion amplification algorithm, micro-vibration analysis is performed on the acquired image sequence to obtain the vibration response waveform of key parts of the target vehicle under acoustic wave impact, and signal analysis is performed to extract the modal features of vibration.

[0105] By comparing the core spectral characteristics with the modal characteristics of vibration, the consistency of acoustic and vibration characteristics is verified, and the final target vehicle is located.

[0106] It is important to note that evidence is collected simultaneously. Once the horn-honking event is determined in step three, the following parallel operations are immediately executed: Acoustic evidence storage: The original acoustic signal that triggered this determination is stored and bound to the horn-honking event record; Visual evidence enhancement acquisition: Based on the latest coordinates of the initially locked vehicle, the high-definition camera is controlled to adjust the gimbal angle and zoom to capture close-up shots of key parts of the target vehicle (such as the front grille and hood) at a high frame rate (e.g., no less than 100 fps), obtaining an image sequence lasting several seconds, which is then stored in association with the horn-honking event record. The control commands include: Pointing parameters: Based on the latest spatiotemporal coordinate sequence corresponding to the initially locked vehicle identifier, the required gimbal pitch and horizontal angles, as well as zoom parameters, are calculated to enable the camera to quickly align with the target vehicle; Focus area: The key parts of the target vehicle (such as the front grille, hood, and license plate) are specified for close-up shooting, obtaining an image sequence lasting several seconds (e.g., 2-3 seconds, covering the period before and after the horn-honking event), which is then stored in association with the horn-honking event record.

[0107] Acoustic feature extraction. The stored raw acoustic signal is preprocessed (e.g., DC component removal, pre-emphasis), and then spectral feature extraction is performed. The extraction methods include: performing a short-time Fourier transform on the preprocessed raw acoustic signal to obtain its spectrogram; extracting the core spectral features that can characterize the specific whistle sound from the spectrogram. These features include, but are not limited to: fundamental frequency trajectory, formant frequency and bandwidth, spectral centroid and roll-off point, and harmonic energy distribution.

[0108] Micro-vibration analysis. The acquired image sequence is preprocessed (e.g., grayscale conversion, image denoising, image stabilization) to eliminate camera shake and background interference. Regions of interest (ROIs) are selected and tracked at key vehicle locations to compensate for macroscopic motion. Based on a phase motion amplification algorithm, the image sequence is decomposed into multiple scales (e.g., using complex manipulable pyramid decomposition) to extract time-varying phase information sensitive to micro-motion. Based on the extracted core spectral features, a target frequency band is determined, covering the main energy distribution of the horn sound and considering the vibration frequencies of vehicle panel structures it may excite. The phase information of this frequency band is linearly amplified and the image sequence is reconstructed. The curve of the average brightness of the RIO over time is extracted from the amplified image sequence as the vibration response waveform. Signal analysis is performed on this waveform to extract its dominant vibration frequency, modal damping ratio, and vibration energy, among other modal characteristics.

[0109] Acoustic-vibration correlation determination. The extracted core spectral features are compared with the modal features of the vibration to verify their consistency. Specifically, this includes: checking whether the dominant vibration frequency matches the acoustic fundamental frequency or the main energy band; calculating the temporal correlation between the acoustic signal energy envelope and the vibration waveform amplitude envelope; and verifying the physical rationality of the vibration start time and the acoustic signal start time. Based on the comparison results, an acoustic-vibration feature consistency score is calculated. The acoustic-vibration feature consistency score is compared with a preset threshold. If the acoustic-vibration feature consistency score > the first threshold, the acoustic-vibration features are determined to be consistent, the initially identified vehicle is officially confirmed as the final target vehicle, and the horn event record is updated. If the second threshold < the acoustic-vibration feature consistency score < the first threshold, the acoustic-vibration correlation is determined to be weak, and the event is marked as "insufficient acoustic-vibration evidence" for manual review. If the acoustic-vibration feature consistency score < the second threshold, the acoustic-vibration features are determined to be significantly inconsistent, and the suspect vehicle is excluded. For example, setting the first threshold to 0.8 and the second threshold to 0.6, the above four indicators are calculated for suspect vehicles with identifiers ID258, ID260, and ID261, respectively. Then, a weighted fusion is performed to obtain the acoustic-vibration feature consistency score, as shown in the following figure. Figure 8 As shown, the horizontal axis represents the suspect vehicle identifier, and the vertical axis represents the acoustic-vibration consistency score. The thick black solid line with solid dots represents the acoustic-vibration feature consistency score (0.94, 0.62, and 0.57 are the scores for the three suspect vehicles, respectively). A solid line with a hollow circle indicates acoustic fundamental frequency matching, a dashed line with a hollow square indicates vibration fundamental frequency consistency, a dotted line with a hollow triangle indicates envelope correlation, and a dotted line with a hollow rhombus indicates the reasonableness of the starting time. The judgment results are represented by the following letters: P indicates pass, i.e., acoustic-vibration feature consistency score ≥ 0.8, indicating highly consistent acoustic-vibration features, and the vehicle is determined to be the final target vehicle in this horn-honking incident; W indicates insufficient evidence, i.e., consistency score 0.6–0.8, indicating weak acoustic-vibration correlation, requiring manual review; F indicates failure, i.e., score < 0.6, indicating significantly inconsistent acoustic-vibration features, and the suspect vehicle is excluded. Therefore, the suspect vehicle with identifier ID258 can be confirmed as the final target vehicle.

[0110] Step 5: Illegal Determination and Evidence Generation. Based on the contextual intelligent analysis model, and according to the audio data, micro-vibration analysis results, and the necessity rules for horn honking, the necessity of the horn honking event is determined, and evidence of illegal horn honking is generated.

[0111] like Figure 9 As shown, this invention provides a contextual intelligence analysis model;

[0112] Specifically:

[0113] Based on the video stream of the monitored area, the traffic situation in the monitored area is quantified in real time at the moment of horn blasting;

[0114] The core spectral features, vibration modal features, motion state vectors of the final target vehicle, vehicle interaction status, and traffic situation are spatiotemporally aligned and feature fused.

[0115] Based on the fused information, the driving scenario is reconstructed and matched with the horn necessity rule to determine whether there is an emergency situation that allows horn use, and the scenario matching result is obtained.

[0116] Based on the context matching results and the acoustic characteristics of the horn blast, a final determination of illegality is made, and evidence of illegal horn blasting is generated.

[0117] It is important to note the quantification of traffic situation. Based on the visual detection and tracking results of step one, the traffic situation in the monitored area is quantified in real time within the time window before and after the timestamp of the horn-honking event. The traffic situation includes the road clearance ahead, the density of nearby vehicles, and the conflict distance between pedestrians / non-motorized vehicles and the target vehicle. The quantification calculation of traffic situation includes: the road clearance ahead, which is the occupancy rate of the passable area ahead of the final target vehicle; the density of nearby vehicles, which is the number of vehicles within a preset radius around the target vehicle; and the conflict distance between pedestrians / non-motorized vehicles and the target vehicle, which can be calculated by identifying and tracking the pedestrians and non-motorized vehicles sensed in step one, and calculating the minimum Euclidean distance between the spatiotemporal coordinates of these traffic participants and the final target vehicle at the moment of horn-honking. The smaller the distance, the higher the potential conflict risk.

[0118] Multimodal feature fusion. All features are unified onto the same timeline based on the timestamp of the horn event, ensuring that the core spectral features, vibration modal features, the motion state vector of the final target vehicle at the moment of horn honking, vehicle interaction status, and traffic situation quantification indicators all correspond to the same moment or the same short period of time, and the aligned features are combined into a contextual feature vector.

[0119] Context reconstruction and rule matching. Based on the context feature vector, the current scenario is categorized into predefined driving context categories using predefined logical rules, such as "an obstacle is cutting in front and the distance is too close," "a pedestrian suddenly appears in the blind spot at the intersection," and "following normally and the road is clear." Then, the reconstructed driving context is matched against a rule base for horn necessity. This rule base is based on traffic regulations and safe driving guidelines, and the rules include: horn honking is allowed in the context of "a pedestrian or non-motorized vehicle suddenly enters the lane from a blind spot, and the collision distance is less than the absolute safety threshold (e.g., 3 meters), and braking cannot avoid a collision"; horn honking is allowed in the context of "a vehicle is reversing, not noticing a target vehicle passing behind, and a collision is imminent"; horn honking is prohibited in contexts such as "the vehicle ahead is starting slowly," "a pedestrian is crossing normally at a crosswalk," and "road is congested and driving slowly." If the current context matches a rule that allows horn honking, the context matching result is "necessary horn honking"; otherwise, the result is "unnecessary horn honking."

[0120] Final Judgment and Evidence Generation. The final violation determination is based on the context matching result: if the context matching result is "unnecessary honking," the honking behavior is directly determined to be an illegal honking event and marked as "illegal"; if the context matching result is "necessary honking," the acoustic characteristics of the honking sound (such as whether the sound pressure level far exceeds the required safety warning) are reviewed. If improper use is found, it is still determined to be an illegal honking event. When determined to be "illegal," a structured illegal honking capture evidence package is automatically generated. This evidence package includes, but is not limited to: basic event information, honking event identifier, timestamp, and identifier of the final target vehicle; evidence images / video clips, including close-up images of the final target vehicle (with its vehicle identifier and spatiotemporal coordinates added to the images), and panoramic video clips of the scene several seconds before and after the honking moment (showing the traffic situation); data evidence, including visual evidence, acoustic evidence, sound vibration consistency judgment results, context matching results and the rule entries they are based on, and a summary of core spectral features (such as fundamental frequency value, duration, and maximum sound pressure level); and an evidence integrity check code to ensure the integrity and immutability of the evidence chain.

[0121] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A traffic horn-honking capture method based on acoustic localization and image coordination, characterized in that, include: Visual detection and tracking: Real-time detection and tracking of vehicles within the monitored area, generating a list of visual targets; The list of visual targets includes the motion trajectory of at least one suspect vehicle and its spatiotemporal coordinate sequence. Acoustic verification: Based on the visual target list, the microphone array is controlled to locate the location of the suspect vehicle by sound waves and obtain the sound source location information; Comprehensive confidence calculation: Spatial matching and verification of the sound source location information with the movement trajectory of the suspect vehicle is performed to calculate the confidence of the horn honking event, determine the occurrence of the horn honking event, and initially locate the target vehicle; Dynamic evidence association: When a horn honking event is determined, the horn audio data and its timestamp are recorded, and the camera equipment is controlled in parallel to collect image information of the area where the target vehicle is located, and the image information is analyzed for micro-vibration. Illegal horn detection and evidence generation: Based on the contextual intelligent analysis model, the necessity of horn ... The construction method of the contextual intelligence analysis model includes: G1. Based on the video stream of the monitored area, the traffic situation in the monitored area is quantified in real time at the moment of horn blasting; the traffic situation includes the smoothness of the road ahead, the density of nearby vehicles, and the conflict distance between pedestrians and non-motorized vehicles and the target vehicle. G2, perform spatiotemporal alignment and feature fusion of core spectral features, vibration modal features, motion state vector of the final target vehicle, vehicle interaction state, and traffic situation; G3. Based on the fused information, the driving situation is reconstructed and matched with the horn necessity rule to determine whether there is an emergency situation that allows horn use, and the situation matching result is obtained. G4. Based on the context matching results and combined with the acoustic characteristics of this horn blast, make a final determination of illegality and generate evidence of illegal horn blasting. The visual detection and tracking steps include: S1. Perform real-time decoding and image preprocessing on the video stream of the monitored area to identify the initial bounding boxes of all vehicles in the image; S2. Assign a unique identifier to each detected vehicle, predict and update the position of each vehicle in consecutive video frames based on the tracking algorithm, convert the pixel coordinates in the image sequence in real time into a spatiotemporal coordinate sequence that is consistent with the microphone array, and form a motion trajectory. S3. Perform semantic analysis on the attributes and dynamic behavior of the detected vehicles to identify their interaction status with the surrounding environment; the interaction status includes normal following, accelerating to approach the vehicle in front, forced lane change, and the presence of an obstacle to cut in front. S4. Based on the active area of ​​potential sound sources calculated in real time by the microphone array, combined with the semantics of vehicle dynamic behavior, vehicles that show potential horn-honking motivations are marked as suspect vehicles, and a list of visual targets is output.

2. The traffic horn-honking capture method based on acoustic localization and image coordination according to claim 1, characterized in that: The method for obtaining the sound source location information includes: M1 receives and parses the list of visual targets, and controls the microphone array to dynamically generate at least one directional detection beam, focusing on the direction of the suspect vehicle; M2. Separate, enhance, and perform feature analysis on the original acoustic signals in each detection beam channel to identify acoustic signals that match the characteristics of a whistle. M3. For acoustic signals that pass the initial screening, calculate their sound source location coordinates and associate them with the vehicle identifier that triggered the calculation to generate sound source location information guided by visual targets.

3. The traffic horn-honking capture method based on acoustic localization and image coordination according to claim 2, characterized in that: The method for obtaining the sound source location information includes correction of the original acoustic signal under adverse weather conditions: N1. Real-time acquisition of meteorological data for the monitored area, including wind speed, wind direction, temperature, humidity, and rainfall; N2. Based on real-time meteorological data, dynamically calculate the speed of sound in the current environment and establish a sound wave propagation model that includes the influence of wind speed vectors in order to correct the basic physical parameters in the sound wave localization algorithm. N3. Based on the sound wave propagation model, construct a wind noise model and perform wind noise filtering on the original acoustic signal of the microphone array at the front end of acoustic signal processing.

4. The traffic horn-honking capture method based on acoustic localization and image coordination according to claim 3, characterized in that: The sound wave propagation model dynamically and continuously calculates the time difference of sound waves propagating from any point in space to each microphone based on the received real-time meteorological data; the wind noise model predicts the statistical characteristics of the noise generated by the wind on the microphones and its spatial correlation in the microphone array based on the received real-time wind speed and wind direction data and combined with the physical configuration of the microphone array, and generates a wind noise reference signal.

5. The traffic horn-honking capture method based on acoustic localization and image coordination according to claim 1, characterized in that: The steps for calculating the overall confidence level include: P1. Map the sound source location coordinates and the movement trajectory of the suspect vehicle to the same spatiotemporal reference system; P2. Based on the motion trajectory of the suspect vehicle, calculate its spatial overlap with the location of the sound source in a continuous time series, and analyze the spatiotemporal variation characteristics of the overlapping area. P3. Extract the motion pattern of the sound source location information and perform motion consistency analysis with the motion trajectory of the suspect vehicle to obtain the motion consistency analysis results. P4. Combine the dynamic behavioral semantics and acoustic signal features of the suspect vehicle to construct a joint visual-acoustic vector; P5. Based on the spatial overlap, motion consistency analysis results, and the fusion results of the audio-visual joint vector, the comprehensive confidence of the horn-honking event is calculated using a weighted average. P6. Compare the overall confidence level with the adaptive threshold to determine the occurrence of the horn honking event and initially locate the target vehicle.

6. The traffic horn-honking capture method based on acoustic localization and image coordination according to claim 5, characterized in that: The adaptive threshold is automatically adjusted based on weather conditions and vehicle density.

7. The traffic horn capture method based on acoustic localization and image coordination according to claim 1, characterized in that: The dynamic evidence association step includes: Q1. When a horn-honking event is determined to have occurred, the original acoustic signal that triggered the determination is stored; the camera equipment is controlled in parallel to acquire image sequences of key parts of the target vehicle; the key parts include the front grille and the hood; Q2. Preprocess the stored raw acoustic signal to extract its core spectral features; Q3. Based on the phase motion amplification algorithm, perform micro-vibration analysis on the acquired image sequence to obtain the vibration response waveform of the key parts of the target vehicle under acoustic wave impact, and perform signal analysis to extract the modal features of the vibration. Q4. Compare the core spectral features with the modal features of vibration to verify the consistency of acoustic and vibration features and pinpoint the final target vehicle.

8. A traffic horn-hitting capture device based on acoustic localization and image coordination, using the traffic horn-hitting capture method based on acoustic localization and image coordination as described in any one of claims 1-7, characterized in that, include: Visual acquisition unit: used to detect and track vehicles in the monitored area in real time and generate a list of visual targets; The list of visual targets includes the motion trajectory of at least one suspect vehicle and its spatiotemporal coordinate sequence. Acoustic acquisition unit: connected to the visual acquisition unit, used to control the microphone array to locate the location of the suspect vehicle by sound waves based on the visual target list, and to obtain the location information of the sound source; Central Processing Unit: Connected to the visual acquisition unit and the acoustic acquisition unit respectively, it is used to spatially match and verify the sound source location information with the movement trajectory of the suspect vehicle, calculate the confidence level of the horn honking event, determine the occurrence of the horn honking event, and initially lock the target vehicle. And when a horn blasting event occurs, it records the horn audio data and its timestamp, controls the camera equipment in parallel to collect image information of the area where the target vehicle is located, and performs micro-vibration analysis on the image information; And it is used for context-based intelligent analysis models to make necessity judgments on horn-honking events based on audio data, micro-vibration analysis results and horn-honking necessity rules, and generate evidence of illegal horn-honking. The construction method of the contextual intelligence analysis model includes: G1. Based on the video stream of the monitored area, the traffic situation in the monitored area is quantified in real time at the moment of horn blasting; the traffic situation includes the smoothness of the road ahead, the density of nearby vehicles, and the conflict distance between pedestrians and non-motorized vehicles and the target vehicle. G2, perform spatiotemporal alignment and feature fusion of core spectral features, vibration modal features, motion state vector of the final target vehicle, vehicle interaction state, and traffic situation; G3. Based on the fused information, the driving situation is reconstructed and matched with the horn necessity rule to determine whether there is an emergency situation that allows horn use, and the situation matching result is obtained. G4. Based on the context matching results and combined with the acoustic characteristics of this horn blast, make a final determination of illegality and generate evidence of illegal horn blasting. The visual detection and tracking steps include: S1. Perform real-time decoding and image preprocessing on the video stream of the monitored area to identify the initial bounding boxes of all vehicles in the image; S2. Assign a unique identifier to each detected vehicle, predict and update the position of each vehicle in consecutive video frames based on the tracking algorithm, convert the pixel coordinates in the image sequence in real time into a spatiotemporal coordinate sequence that is consistent with the microphone array, and form a motion trajectory. S3. Perform semantic analysis on the attributes and dynamic behavior of the detected vehicles to identify their interaction status with the surrounding environment; the interaction status includes normal following, accelerating to approach the vehicle in front, forced lane change, and the presence of an obstacle to cut in front. S4. Based on the active area of ​​potential sound sources calculated in real time by the microphone array, combined with the semantics of vehicle dynamic behavior, vehicles that show potential horn-honking motivations are marked as suspect vehicles, and a list of visual targets is output.

Images