A multi-source data fusion and intelligent analysis method of a smart traffic enterprise

Abstract

The application discloses a kind of multi-source data fusion and intelligent analysis method of wisdom traffic enterprise, it is related to wisdom traffic technical field, the specific steps of this method are as follows: S100 real-time acquisition traffic enterprise's video frame data, radar trajectory data and voiceprint data, S200 calculates quantized video definition score, S300 calculates radar signal-to-noise ratio, S400 calculates voiceprint environmental interference index, S500, weight distribution and data fusion, the present application can accurately judge video, radar, voiceprint data quality by real-time evaluation and credibility quantization of multi-modal data quality, in complex environment, according to the quality evaluation result dynamic adjustment cross-modal attention weight matrix, that is, when video quality declines, the system automatically increases radar data weight, to ensure that always focus on high-quality data source key information, learn more accurate, comprehensive abnormal characteristics, effectively reduce traffic accident incidence, guarantee road traffic safety.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to a method for multi-source data fusion and intelligent analysis for intelligent transportation enterprises. Background Technology

[0002] With the acceleration of urbanization and the continuous growth of traffic flow, intelligent transportation systems have become a key infrastructure for modern urban development. Efficient and precise technologies are urgently needed to address increasingly complex traffic conditions. Multi-source data fusion and intelligent analysis technologies, as the core support for intelligent transportation, aim to integrate multiple data sources and uncover the potential value behind the data, thereby achieving goals such as traffic flow optimization, accident prevention, and improved travel efficiency. Data sources such as video surveillance, radar detection, and voiceprint collection are widely used in the field of intelligent transportation because they can reflect traffic conditions from different dimensions.

[0003] However, existing multi-source data fusion and analysis methods lack real-time and effective means for data quality assessment. Different data sources are affected by factors such as environment and equipment aging, resulting in large fluctuations in data quality. For example, in severe weather, video surveillance images are easily blurred, and existing methods cannot quantify the degree of video quality degradation in a timely and accurate manner, leading to low-quality video data participating in fusion analysis and seriously interfering with the accuracy of the results. In cross-modal data processing, traditional methods struggle to flexibly focus on key information in complex environments. When there is insufficient light at night or severe weather such as heavy rain or fog, traditional cross-modal processing mechanisms cannot quickly adjust their focus on video, radar, and voiceprint data, resulting in a significant reduction in the ability to detect abnormal events and failing to meet the needs of intelligent transportation for efficient processing of complex scenarios.

[0004] In summary, current multi-source data fusion and analysis technologies in the field of intelligent transportation have shortcomings in real-time data quality assessment and flexibility in cross-modal data processing. There is an urgent need for an innovative method that can assess the quality of multi-source data in real time and accurately, and intelligently focus on key information in complex environments to improve the ability to detect abnormal events, so as to meet the growing management needs of intelligent transportation systems. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a multi-source data fusion and intelligent analysis method for intelligent transportation enterprises. It can accurately judge the quality of video, radar, and voiceprint data through real-time evaluation and reliable quantification of multimodal data quality. In complex environments, it can dynamically adjust the cross-modal attention weight matrix based on the quality evaluation results to ensure that it always focuses on key information from high-quality data sources, providing strong support for timely response measures and ensuring road traffic safety.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for multi-source data fusion and intelligent analysis of intelligent transportation enterprises, wherein the specific steps of the method are as follows:

[0007] S100: Real-time acquisition of video frame data, radar trajectory data and voiceprint data from transportation companies, and synchronization of timestamps and spatial coordinates;

[0008] S200. For video data, by analyzing the edge sharpness and contrast characteristics of the video image, a quantified video sharpness score q is calculated. v ;

[0009] S300. Based on the radar data, analyze the ratio of radar echo signal power to noise power, and calculate the radar signal-to-noise ratio q. r ;

[0010] S400. Based on the voiceprint data, taking into account the environmental noise intensity, frequency distribution, and the degree of overlap with the voiceprint signal frequency, the voiceprint environmental interference index q is calculated. a ;

[0011] The S500 uses video clarity score, radar signal-to-noise ratio, and acoustic interference index as initial parameters, obtains initial weights through normalization, and fuses multi-modal data to complete multi-source data fusion and intelligent analysis.

[0012] Furthermore, the video clarity score in S200 Where α and β are adjustment coefficients, and α + β = 1, α = 0.6, β = 0.4. This indicates that the coordinates in the video frame are (x) i ,y i The gradient value of a pixel is used to measure the sharpness of image edges, where n is the total number of pixels in the video frame involved in the gradient calculation, and I(x) is the gradient value of the pixel. j ,y j ) is the coordinate (x j ,y j The pixel grayscale value at () is the average grayscale value of the entire video frame, and m is the number of all pixels in the video frame. The video sharpness score is determined by comprehensively considering the image edge details and overall contrast.

[0013] Furthermore, the radar signal-to-noise ratio in the S300 Among them, P s P represents the power of the target reflected signal received by the radar. n This is the sum of the power of the radar's internal noise and the ambient noise, used to quantify the ratio of effective signal to noise received by the radar, in order to evaluate the quality of radar trajectory data.

[0014] Furthermore, in S400, the voiceprint interference index is obtained by transforming the collected voiceprint signal to obtain its time-frequency domain representation S(t,f), and the interference index is defined. Where T is the total duration of the voiceprint signal, and F n For the noise frequency range, F t This index represents the full frequency range of the voiceprint signal and reflects the proportion of noise components in the voiceprint signal. It is used to assess the degree of environmental interference affecting voiceprint data.

[0015] Furthermore, the specific steps of S500 are as follows:

[0016] Get video clarity rating q v Radar signal-to-noise ratio q r and voiceprint interference index q a ;

[0017] For q v q r q a Normalization is performed to obtain weight coefficients that can be used for fusion. Where q j =q v +q r +q a q i q v q r q a The input is used to make the sum of all weights equal to 1, thus obtaining the initial weights w of the video data. v The initial weights w of the radar data r The initial weights w of the voiceprint data a ;

[0018] A weighted fusion algorithm is used to fuse multimodal data.

[0019] Furthermore, during the fusion of multimodal data using the attention-based weighted fusion algorithm, video data V, radar data R, and voiceprint data A are acquired via S100, and the fused data F = w v V+w r R+w a A, where F represents the fused multi-source data, which is the comprehensive data ultimately used for intelligent analysis, w v w r w aThese are the weights corresponding to video data, radar data, and voiceprint data, reflecting the importance of each modality in the fusion process. V, R, and A are the original video data, radar data, and voiceprint data, respectively. Through weighted fusion, high-quality data plays a role in the fusion result, while low-quality data is controlled, thereby optimizing the fusion effect of multi-source data.

[0020] Furthermore, the S500 introduces an attention mechanism, using data features to calculate an attention score, i.e., the video data features after feature extraction from multimodal data are V. f Radar data characteristics are R f The voiceprint data feature is A f Attention scores for each modality of data are calculated using a scoring function, and attention scores for the video data are also calculated. Attention score of radar data Attention score of voiceprint data Where θ is a learnable parameter vector, the importance of different data features to the final result is explored based on the fusion task. These represent the transposes of the video data feature vector, radar data feature vector, and speaker data feature vector, respectively. After obtaining the attention score, the score is converted into a weight score, i.e. The weighted scores are combined with the initial weights to obtain... λ is a balance coefficient with a value of [0, 1], which is used to adjust the influence of the initial weights and the weights after the attention mechanism is adjusted, and to optimize the weight allocation.

[0021] Furthermore, when the quality of modal data changes abruptly, i.e. at night or in severe weather, the S500 immediately triggers an emergency reallocation of attention weights. Through a dynamic calibration mechanism, it ensures that the cross-modal attention mechanism can always detect abnormal events based on high-quality data.

[0022] Compared with existing technologies, this method for multi-source data fusion and intelligent analysis in smart transportation enterprises has the following advantages:

[0023] I. This invention can accurately judge the quality of video, radar, and voiceprint data through real-time multimodal data quality assessment and reliable quantification. In complex environments, it can dynamically adjust the cross-modal attention weight matrix based on the quality assessment results. That is, when the video quality deteriorates, the system automatically increases the radar data weight to ensure that it always focuses on the key information of high-quality data sources, learns more accurate and comprehensive abnormal features, effectively reduces the incidence of traffic accidents, and ensures road traffic safety.

[0024] Second, the pre-support and dynamic calibration mechanism of this invention provides dual protection for traffic operation. The pre-support sets initial weights based on prior knowledge of data quality and allocates attention to different data sources to reduce judgment errors in the initial stage. The dynamic calibration monitors sudden changes in data quality in real time, triggering weight reallocation. This mechanism ensures that the system can quickly adjust its data processing strategy under various emergencies and always analyze based on high-quality data, which greatly improves the adaptability and stability of intelligent transportation and reduces the risk of failure due to environmental changes or equipment failure.

[0025] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0027] Figure 1 This is an operational diagram of a multi-source data fusion and intelligent analysis method for a smart transportation enterprise.

[0028] Figure 2 This is a flowchart of the steps for user classification in Example 2. Detailed Implementation

[0029] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0030] Example 1

[0031] like Figure 1 As shown, this embodiment focuses on a multi-source data fusion and intelligent analysis method for intelligent transportation enterprises. It elaborates in detail how to evaluate data quality, perform data fusion and weight adjustment by real-time acquisition of video frames, radar trajectories and voiceprint data, so as to achieve efficient abnormal event detection in complex environments, thereby improving the level of intelligent transportation management and providing strong support for the safe and efficient operation of urban transportation.

[0032] In its implementation, data acquisition equipment is deployed, including high-definition cameras for acquiring video frame data, radar for acquiring radar trajectory data, and microphone arrays for collecting voiceprint data. These devices work together to collect traffic data in real time. The high-definition cameras continuously capture traffic footage at a certain frame rate, generating a series of video frames. The cameras are precisely calibrated to ensure that the shooting angle covers the traffic area. The radar emits millimeter-wave signals and determines the target's position, speed, and direction of motion by receiving the echo signals reflected from the target object, thus forming radar trajectory data and ensuring the accuracy and stability of the data. The microphone arrays are distributed at intersections to collect various sounds generated during vehicle movement, including engine sounds, braking sounds, and horn sounds. These sound signals are converted into digital signals and used as voiceprint data for subsequent processing. To ensure the spatiotemporal consistency of multi-source data, precise timestamps and spatial coordinates are added to the acquired video frame data, radar trajectory data, and voiceprint data. The timestamps mark the specific time of data acquisition, and the spatial coordinates are based on the radar's geographic coordinate system to determine the location of the data acquisition point on Earth, enabling accurate matching of data from different data sources in both time and space.

[0033] The collected data is affected by various factors, resulting in inconsistent quality. Therefore, it is necessary to conduct quality assessments on video, radar, and voiceprint data separately to provide a reliable basis for subsequent data fusion. Among these:

[0034] Video sharpness score calculation: A certain number of pixels are selected from the captured video frames to calculate the video sharpness score. For each selected pixel (x... i ,y i ), by calculating its gradient value To measure image edge sharpness, the gradient value reflects the degree of brightness change around a pixel; the greater the change, the sharper the edge. Simultaneously, the grayscale value I(x) of all pixels in the entire video frame is calculated. j ,y j ), and calculate the average gray value. According to the formula The video sharpness score q is determined by taking into account both image edge detail and overall contrast. v , where α and β are adjustment coefficients, and α+β=1. In this embodiment, they are used to balance the influence of edge sharpness and overall contrast on the video sharpness score, n is the total number of pixels involved in the gradient calculation, and m is the total number of pixels in the video frame.

[0035] Radar signal-to-noise ratio calculation: Analyze the radar echo signal and denote the power of the received target reflected signal as P. s The sum of the radar's internal noise and the ambient noise power is denoted as P. n,according to Calculate the radar signal-to-noise ratio q r The signal-to-noise ratio (SNR) is used to quantify the ratio of effective signal to noise received by the radar. The higher the SNR, the stronger the effective signal received by the radar and the higher the quality of the radar trajectory data.

[0036] Voiceprint environmental interference index calculation: The collected voiceprint signal is transformed to obtain its time-frequency domain representation S(t,f), and the total duration T of the voiceprint signal and the noise frequency range F are determined. n and the full frequency range F of the voiceprint signal t ,according to Calculate the ambient interference index q of the soundprint a This index reflects the proportion of noise components in the voiceprint signal. The higher the proportion, the greater the environmental interference with the voiceprint data and the lower the data quality.

[0037] Based on the data quality assessment results, multimodal data fusion is performed, and the weights of each modality in the fusion process are calculated. The process is as follows: Obtain the video clarity score q. v Radar signal-to-noise ratio q r and voiceprint environmental interference index q a Then, to ensure the sum of all weights is 1 for easier subsequent fusion calculations, they are normalized according to the formula. Calculate the initial weights w of the video data respectively. v0 Initial weights w of radar data r0 The initial weights w of the voiceprint data a0 , where q j =q v +q r +q a q i q v q r q a Any one of the acquired video data V, radar data R, and voiceprint data A is used in an attention-based weighted fusion algorithm to extract features from the multimodal data, resulting in video data features V. f Radar data characteristics R f and voiceprint data features A f The attention score for each modality is calculated using a scoring function. For video data, the attention score is... For radar data, attention score For voiceprint data, attention score Here, θ is a learnable parameter vector. Its value is continuously adjusted through training according to the requirements of the fusion task, in order to uncover the importance of different data features to the final fusion result. Let represent the transposes of the video data feature vector, radar data feature vector, and speaker data feature vector, respectively. After obtaining the attention score, convert it into a weight score using the formula: Finally, the weight scores are combined with the initial weights to obtain the final weights used for data fusion. Where λ is the balance coefficient, ranging from [0, 1], used to adjust the influence of the initial weights and the weights adjusted by the attention mechanism. When λ is close to 1, the influence of the initial weights is greater; when λ is close to 0, the influence of the weights adjusted by the attention mechanism is greater, in order to achieve the best data fusion effect. Based on the calculated weights, the multimodal data is fused, and the fused data F = w v V+w r R+w a A. This fused data integrates information from video, radar, and voiceprint data as the final comprehensive data used for intelligent analysis. This allows high-quality data to play a greater role in the fusion results, effectively controls the impact of low-quality data, and optimizes the fusion effect of multi-source data.

[0038] The fused data is input into an anomaly detection model to determine whether an anomaly has occurred, and appropriate handling measures are taken based on the event type and location. This embodiment uses a deep learning-based neural network model as the anomaly detection model. This model has been trained on a large amount of historical traffic data to learn the characteristic patterns of normal traffic conditions and various anomalies. During training, the fused multi-source data is used as input, and the traffic state (normal or abnormal, and the specific type of the anomaly event) is labeled. The model continuously adjusts its internal parameters to make the prediction results close to the labeled results. After training, the model has the ability to detect anomalies on new input data. The fused data is then input into the training... In a good anomaly detection model, the model analyzes and judges the data. If the model output indicates that the current traffic state is abnormal, it further determines the type of the abnormal event, such as traffic accidents, traffic congestion, vehicle violations, etc., and determines the location of the event through spatial coordinate information in the data. Once an abnormal event is detected, the corresponding processing procedure is immediately initiated. For traffic accidents, alarm information is automatically sent to traffic management departments and emergency centers. For traffic congestion, real-time road condition information is released through the traffic guidance system to guide vehicles to avoid congested road sections and optimize traffic flow. For vehicle violations, relevant evidence, such as video screenshots and radar trajectory data, is recorded, and the violation information is stored and transmitted for subsequent processing.

[0039] In real-world traffic environments, data quality can change abruptly. For example, insufficient light at night can degrade video quality, or severe weather can affect the quality of radar and voiceprint data. To ensure that anomaly detection is always based on high-quality data, this embodiment employs a dynamic calibration mechanism. This mechanism monitors changes in video clarity scores, radar signal-to-noise ratio (SNR), and voiceprint environmental interference index in real time. If a sudden change in the quality of one modality is detected, such as a significant decrease in video clarity scores at night, or a decrease in radar SNR or an increase in the voiceprint environmental interference index due to heavy rain, the dynamic calibration mechanism is immediately triggered. The processing flow of this dynamic calibration mechanism is as follows: when a sudden change in data quality is detected, the weights of each modality are recalculated based on the current data quality assessment results. This involves recalculating the weight coefficients, adjusting the weights based on an attention mechanism, and fusing the weights to obtain new weights. These new weights are then used to re-fuse the multimodal data, ensuring that the fused data more accurately reflects the current traffic conditions and provides a reliable basis for anomaly detection. Through this dynamic calibration mechanism, the key information from high-quality data sources can be consistently focused on in various complex environments, improving the accuracy and reliability of anomaly detection.

[0040] In summary, this embodiment details the complete implementation process of a multi-source data fusion and intelligent analysis method for intelligent transportation enterprises. By acquiring video frames, radar trajectories, and voiceprint data, and synchronizing timestamps and spatial coordinates, quantitative evaluation is performed to accurately determine data quality. Based on the evaluation results, weights are calculated and weighted fusion is performed to optimize the fusion effect. Abnormal events are detected and corresponding measures are taken. When data quality changes abruptly, weights are adjusted in a timely manner to ensure stability and accuracy. This achieves efficient fusion and intelligent analysis of multi-source data, effectively solving the shortcomings of existing technologies in real-time data quality assessment and cross-modal data processing flexibility. In complex traffic environments, it can accurately focus on key information from high-quality data sources, improve the ability to detect abnormal events, and provide strong support for the efficient operation of intelligent transportation and road traffic safety.

[0041] Example 2

[0042] like Figure 2 As shown in the first embodiment, this embodiment focuses on the multi-source data fusion and intelligent analysis method for intelligent transportation enterprises. It details how, in urban traffic scenarios, by collecting video frames, radar trajectories, and voiceprint data, evaluating data quality, and utilizing fusion strategies and intelligent analysis methods, accurate monitoring of traffic conditions and efficient handling of abnormal events can be achieved, thereby improving the level of intelligence in traffic management and road safety.

[0043] First, the data acquisition phase (S100) begins, where high-definition cameras, radar, and voiceprint collection points are installed to collect video frame data, radar trajectory data, and voiceprint data from transportation companies in real time. To ensure consistency of multi-source data in time and space, precise timestamps and spatial coordinates are added to each set of collected data, enabling data from different data sources to be processed and analyzed within a unified spatiotemporal framework.

[0044] Then, the data quality assessment stage (S200) is entered, where the quality of the collected data is assessed. For video data, the contrast of the image is analyzed; for radar data, the stability and accuracy of the signal are evaluated; and for voiceprint data, the degree of interference from environmental noise is the focus.

[0045] Subsequently, the data fusion and weight determination stage (S300) is entered. Based on the data quality assessment results, different types of data are fused and their weights in the fusion process are determined. During the data fusion process, video data, radar trajectory data, and voiceprint data are weighted and fused according to the weights of the data. In this way, high-quality data can play a greater role in the fusion result, while the impact of low-quality data on the final result will be weakened, thereby improving the effect of data fusion.

[0046] Next, the abnormal event detection and handling stage (S400) is entered. The fused data is used to train the traffic anomaly model, which can accurately identify various normal and abnormal traffic patterns. When an abnormal change in traffic flow is detected, it will be judged that there is an abnormal event such as traffic congestion or accident. Once an abnormal event is detected, the corresponding handling process will be initiated immediately.

[0047] Finally, the system enters the dynamic calibration and optimization phase (S500). In actual operation, the traffic environment is constantly changing, and the data quality will fluctuate accordingly. In order to ensure that the system can always accurately analyze traffic conditions and monitor changes in data quality in real time, once a sudden change in data quality is detected, the data quality is re-evaluated, and the weights of different data are readjusted based on the new evaluation results. This dynamic calibration mechanism can always maintain accurate monitoring of traffic conditions and efficient detection of abnormal events in a complex and ever-changing traffic environment.

[0048] In summary, this embodiment demonstrates in detail the application of multi-source data fusion and intelligent analysis methods of intelligent transportation enterprises in urban transportation hubs. Through comprehensive data collection, rigorous data quality assessment, reasonable data fusion and weight determination, efficient anomaly detection and handling, and dynamic calibration and optimization mechanisms, it achieves accurate monitoring and effective management of complex traffic conditions. This method can promptly detect and handle abnormal traffic events, improve road traffic efficiency, and ensure traffic safety.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1.A method for multi-source data fusion and intelligent analysis of intelligent transportation enterprises, characterized in that, The specific steps of the method are: S100, real-time collection of traffic enterprise video frame data, radar trajectory data and voiceprint data, and synchronization of time stamp and space coordinate; S200. For video data, by analyzing the edge sharpness and contrast characteristics of the video image, a quantified video sharpness score q is calculated. v ; S300, for the radar data, analyze the ratio of radar echo signal power and noise power, calculate the radar signal to noise ratio q r ; S400, for voiceprint data, comprehensive consideration of environmental noise intensity, frequency distribution and the degree of overlap with voiceprint signal frequency, calculate the voiceprint environmental interference index q a ; S500, taking the video definition score, radar signal-to-noise ratio and voiceprint environmental interference index as the initialization parameter, obtaining the initial weight through normalization processing, introducing the attention mechanism and fusing the multi-modal data to complete the multi-source data fusion and intelligent analysis; The specific steps of S500 are: obtaining a video sharpness score q v , radar signal-to-noise ratio q r , and voiceprint interference index q a ; Normalization is performed on q v , q r , q a to obtain weight coefficients that can be used for fusion, wherein q j =q v +q r +q a , q i represents the input of q v , q r , q a , so that the sum of each weight is 1, thereby obtaining the initial weight w v0 of the video data, the initial weight w r0 of the radar data, and the initial weight w a0 of the voiceprint data. The weighted fusion algorithm based on the attention mechanism is used to fuse the multi-modal data. 2.The method of claim 1, wherein, The video definition score in the S200 wherein, α and β are adjustment coefficients and α+β=1, α=0.6, β=0.4, represents a gradient value of a pixel point with coordinates (x i ,y i ) in a video frame, used for measuring image edge definition, n is a total number of pixel points participating in gradient calculation in the video frame, I(x j ,y j ) is a pixel gray value at coordinates (x j ,y j ), is an average gray value of the entire video frame, m is a number of all pixel points in the video frame, and the video definition score is determined by comprehensively considering image edge details and overall contrast. 3.The method of claim 1, wherein, The radar signal-to-noise ratio in the S300 where P s is the target reflection signal power received by the radar, P n is the sum of the radar internal noise and the ambient noise power, used to quantify the proportion of effective signal and noise received by the radar to assess the quality of the radar track data. 4.The method of claim 1, wherein, The voiceprint interference index in the S400 is obtained by transforming the collected voiceprint signal to obtain a time-frequency domain representation S(t,f), and the interference index is defined as where T is the total duration of the voiceprint signal, F n is the noise frequency range, F t is the full frequency range of the voiceprint signal, and the index reflects the proportion of noise components in the voiceprint signal and is used to evaluate the degree of environmental interference on the voiceprint data. 5.The method of claim 1, wherein, The attention mechanism-based weighted fusion algorithm obtains video data V, radar data R and voiceprint data A through S100 in the process of fusing the multi-modal data, and the fused data F = w v V + w r R + w a A, wherein F represents the fused multi-source data, which is the comprehensive data finally used for intelligent analysis, w v , w r , w a are weights corresponding to the video data, the radar data and the voiceprint data respectively, reflecting the importance of each modal data in the fusion process, V, R and A are the original video data, the original radar data and the original voiceprint data respectively, high-quality data plays a role in the fusion result through the weighted fusion manner, low-quality data is controlled, and thus the multi-source data fusion effect is optimized. 6.The method of claim 5, wherein, The S500 introduces an attention mechanism, and calculates an attention score using data features, that is, the video data features after feature extraction of the multi-modal data are V f , the radar data features are R f , and the voiceprint data features are A f The attention score of each modal data is calculated by a scoring function, and the attention scores of the video data, the radar data and the voiceprint data are respectively calculated as follows Wherein θ is a learnable parameter vector, and according to the fusion task, the importance of different data features to the final result is mined, Respectively represent the transposition of the video data feature vector, the radar data feature vector and the voiceprint data feature vector, after obtaining the attention score, the score is converted into a weight score, that is, The weight score is fused with the initialized weight to obtain ​​ λ is a balance coefficient, the value of which is [0, 1], used to adjust the influence degree of the initial weight and the weight after the attention mechanism adjustment, and optimize the weight distribution. 7.The method of claim 1, wherein, When the quality of the modal data is suddenly changed, i.e. at night and in bad weather, the attention weight emergency redistribution is triggered immediately, and through the dynamic calibration mechanism, it is ensured that the cross-modal attention mechanism can always detect abnormal events based on high-quality data.

Images