High-sensitivity detection method and system for dangerous goods based on multimodal sensor data processing

Abstract

The present invention discloses a high-sensitivity detection method and system for dangerous goods based on multimodal sensor data processing, including S1: capturing electrical signals triggered by dangerous goods molecules through multimodal sensors to generate multimodal data sequences; S2: performing multi-scale decomposition of the multimodal data sequences through AMNSA and then adaptively adjusting the threshold to obtain optimal feature data; S3: taking spatiotemporal feature data as input based on optimized DCAFEN and outputting optimized spatiotemporal features; S4: constructing a spatiotemporal feature fingerprint library based on the optimized spatiotemporal features, and designing a dynamic logic gate judgment mechanism to identify dangerous goods, and outputting a hazard confidence result. Compared with traditional methods, the accuracy of 50-100μV weak signal detection is greatly improved, effectively solving the problem of thermal noise masking weak adsorption features.

Description

Technical Field

[0001] The present invention relates to the technical field of dangerous goods detection, and in particular to a high-sensitivity dangerous goods detection method and system based on multimodal sensor data processing. Background Art

[0002] With the rapid increase in demand for the detection of dangerous goods (such as explosives and biochemical agents) in places such as airports, subways, and large-scale events, traditional single-modal detection technologies (such as X-ray imaging and metal detection) have obvious limitations.

[0003] Currently, existing hazardous materials detection systems (such as single ion mobility spectrometers) are susceptible to thermal noise (carrier thermal motion), 1 / f noise (low-frequency drift), and sudden environmental interference (human flow, air flow, electromagnetic coupling) in complex environments (such as airport security). Traditional wavelet threshold filtering can only process Gaussian noise and has poor suppression effect on non-stationary noise (such as environmental interference with a peak value of 30% of the signal intensity). As a result, the weak characteristics of low-concentration hazardous materials (50ml of liquid corresponds to a 50-100μV signal) are masked, and the missed detection rate increases. Therefore, a high-sensitivity hazardous materials detection method and system based on multimodal sensor data processing is proposed. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned objectives, the present invention proposes the following technical solutions:

[0005] High-sensitivity detection method for dangerous goods based on multimodal sensor data processing, including:

[0006] S1: Capture the electrical signals caused by dangerous goods molecules through multimodal sensors to generate multimodal data sequences;

[0007] S2: After multi-scale decomposition of the multimodal data sequence through AMNSA, adaptive threshold adjustment is performed to obtain the optimal feature data;

[0008] S3: Based on the optimized DCAFEN, the spatiotemporal feature data is taken as input and the optimized spatiotemporal features are output;

[0009] The optimized DCAFEN includes an input layer, a multi-branch feature extraction layer, a dual attention fusion layer, and an output layer;

[0010] S4: Based on the optimized spatiotemporal features, a spatiotemporal feature fingerprint library is constructed, and a dynamic logic gate judgment mechanism is designed to identify dangerous items and output the hazard confidence results.

[0011] The multimodal data sequence includes a time current signal and a frequency intensity signal, and the multimodal sensor includes a portable explosive detector and a 785nm laser.

[0012] The multi-scale decomposition process is as follows:

[0013] AMNSA uses a 5-layer wavelet packet decomposition based on the db4 wavelet basis to decompose the time current signal into 32 frequency bands, and then extracts the quantization coefficients from the decomposed sub-signals to obtain the first detail coefficients and the second detail coefficient ;

[0014] The first and second layers of wavelet packets decompose noise greater than 1kHz, the third and fourth layers of wavelet packets decompose noise from 100Hz to 1kHz, and the fifth layer of wavelet packets decomposes noise less than 100Hz.

[0015] The adaptive threshold control process is:

[0016] Based on the first detail coefficient and the second detail coefficient , get the first detail coefficient and the second detail coefficient The first kurtosis of each frequency band coefficient and the second kurtosis ;

[0017] For any kurtosis greater than or equal to The threshold is multiplied by the correction factor ;

[0018] First kurtosis and the second kurtosis The correction process for the threshold values ​​that are all less than 3.5 is:

[0019] ;

[0020] in, Dynamic adjustment, is the threshold The basic calculation factor of

[0021] After adaptive threshold adjustment, the frequency band coefficients of each mode are reconstructed to obtain the adjusted time current signal and frequency intensity signal , forming the best feature data .

[0022] The multi-branch feature extraction layer implementation process is as follows:

[0023] The time-current signal after adjustment in the best characteristic data , using 1D dilated convolution with kernel size 5 and dilation rate 2 to capture long-distance dependent output time features ;

[0024] Adjusted frequency intensity signal in the best feature data The frequency distribution characteristics of the time spectrum are generated by short-time Fourier transform with a window length of 256 and an overlap rate of 50%. , use 1×1 convolution to compress the number of channels to output frequency domain features .

[0025] The dual attention fusion layer implementation process is as follows:

[0026] Time characteristics , calculate the energy share of each channel through global average pooling , the formula is:

[0027] ;

[0028] Where c is the channel index and t is the time index;

[0029] Frequency domain features Generate spatial weights using 3×3 convolution ;

[0030] The parallel weighted fusion method is used to integrate the time features and frequency domain features, and the channel attention weights are , spatial attention weight and time characteristics , frequency domain characteristics Parallel fusion is performed to obtain the fusion features output by the dual attention fusion layer.

[0031] The output layer implementation process is:

[0032] Fusion features Perform compression adaptation processing and use 1×1 convolution kernel to fusion features Channel compression is performed to reduce the feature dimension from high dimension to 64 dimensions, and the optimized spatiotemporal features are output.

[0033] The construction process of the spatiotemporal feature fingerprint library is as follows:

[0034] Extract the time core features and frequency core features from the optimized spatiotemporal features and encode them as fingerprints;

[0035] The fingerprint includes a time fingerprint and a frequency fingerprint;

[0036] The time core features include rising edge slope, peak timestamp and peak decay rate;

[0037] The frequency core features include frequency coordinates, peak intensity ratio and peak spacing ratio;

[0038] The acquired time fingerprint and frequency fingerprint are combined through knowledge graph technology to form a spatiotemporal feature fingerprint library of dangerous goods.

[0039] The implementation process of the dynamic logic gate determination mechanism is as follows:

[0040] Among them, the dynamic logic gate judgment mechanism includes basic gate, enhancement gate, and veto gate;

[0041] The basic gate time fingerprint condition is that the difference between the rising edge slope of the real-time extracted and the rising edge slope of the spatiotemporal feature fingerprint library is less than The basic gate frequency fingerprint condition is that the difference between the first characteristic peak frequency extracted in real time and the first characteristic peak frequency of the spatiotemporal characteristic fingerprint library is less than ;

[0042] The enhanced gate time fingerprint condition is to extract the peak time stamp in real time and the peak time stamp of the spatiotemporal feature fingerprint database is Within the interval, the enhanced gate frequency fingerprint condition is that the difference between the first peak intensity ratio extracted in real time and the first peak intensity ratio of the spatiotemporal feature fingerprint library is less than 0.1;

[0043] The rejection gate uses a time fingerprint condition when the difference between the peak decay rate extracted in real time and the peak decay rate of the spatiotemporal feature fingerprint library is greater than The rejection gate frequency fingerprint condition is that the number of newly detected unknown characteristic peaks exceeds 1, or the intensity ratio of a single unknown peak exceeds 0.2.

[0044] High-sensitivity dangerous goods detection system based on multimodal sensor data processing, including:

[0045] Data acquisition module: Captures electrical signals caused by dangerous goods molecules through multimodal sensors and generates multimodal data sequences;

[0046] Feature acquisition module: AMNSA is used to perform multi-scale decomposition of multimodal data sequences and then perform adaptive threshold adjustment to obtain optimal feature data;

[0047] Feature optimization module: takes spatiotemporal feature data as input based on the optimized DCAFEN and outputs optimized spatiotemporal features;

[0048] Hazard identification module: Build a spatiotemporal feature fingerprint library based on optimized spatiotemporal features, design a dynamic logic gate judgment mechanism to identify dangerous items, and output the hazard confidence results.

[0049] The present invention has the following beneficial effects:

[0050] 1. Through AMNSA adaptive multi-scale noise suppression technology, 5-layer db4 wavelet packet decomposition is used to achieve fine separation in the time and frequency domains. High-frequency circuit noise (>1kHz), medium-frequency molecular adsorption characteristics (100Hz-1kHz) and low-frequency baseline drift (<100Hz) are layered processed. Combined with dynamic threshold adjustment of kurtosis (threshold correction factor γ=1.2), the signal retention rate of low-concentration hazardous materials is improved, and the environmental noise suppression rate reaches 92%. Compared with traditional methods, the accuracy of weak signal detection of 50-100μV is greatly improved, effectively solving the problem of thermal noise masking weak adsorption characteristics.

[0051] 2. By optimizing the DCAFEN network through 1D dilated convolution (kernel size 5, dilation rate 2) and a dual attention fusion layer (channel attention + spatial attention parallel weighting), the feature dimension is reduced from high to 64 dimensions, and the computational complexity is controlled at O(9TH), adapting to the hardware of portable devices. In addition, the dynamic logic gate judgment mechanism (basic gate, enhancement gate, veto gate) is combined with the spatiotemporal feature fingerprint library to achieve high-precision matching of dangerous goods. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a method step diagram of the highly sensitive dangerous goods detection method and system based on multimodal sensor data processing proposed by the present invention.

[0053] Figure 2 This is a system block diagram of the highly sensitive dangerous goods detection method and system based on multimodal sensor data processing proposed by the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 As shown, the highly sensitive dangerous goods detection method based on multimodal sensor data processing proposed by the present invention includes:

[0057] S1: Capture the electrical signals caused by dangerous goods molecules through multimodal sensors to generate multimodal data sequences;

[0058] The multimodal data sequence includes a time current signal and a frequency intensity signal, and the multimodal sensor includes a portable explosive detector and a 785nm laser;

[0059] Use portable explosive detectors to achieve multimodal molecular signal conversion of dangerous goods based on ion mobility spectrometry (IMS) and Raman spectroscopy;

[0060] After the molecules of dangerous goods (such as explosives) are sampled by the portable explosives detector, they are ionized by the ionization source ( Radioactive source, photoionization source) ionizes into ions, and when the ions reach the detector, a current signal is generated, which outputs a time current signal. ;

[0061] Use a laser light source (785nm laser) to irradiate the molecules of dangerous goods, triggering molecular vibration and rotational energy level transitions, generating Raman scattered light. The difference between the scattered light frequency and the incident light frequency corresponds to the specific vibration mode of the molecule. The scattered light is collected by a spectral detector and a frequency intensity signal is output. ;

[0062] Output time current signal and frequency intensity signal It constitutes a multimodal data sequence.

[0063] S2: After multi-scale decomposition of the multimodal data sequence through AMNSA, adaptive threshold adjustment is performed to obtain the optimal feature data;

[0064] The process of multi-scale decomposition is:

[0065] Portable explosive detectors face three main types of noise interference in actual operation:

[0066] Thermal noise: Produced by the irregular thermal motion of carriers, it can mask microvolt-level signal changes caused by low-concentration hazardous substances.

[0067] 1 / f noise: This is related to sensor material defects and is dominant in the low-frequency range (<1kHz). Its power spectral density increases as the frequency decreases, causing signal drift during long-term detection.

[0068] Environmental interference noise: This includes airflow disturbances caused by the movement of people in security inspection scenarios and electromagnetic coupling caused by the proximity of metal objects. This type of noise is characterized by non-stationary and sudden occurrences.

[0069] When 50ml of dangerous liquid remains at the bottom of the container, the response signal amplitude of the portable explosive detector is approximately , and the peak value of environmental noise can reach 30% of the signal strength. Although traditional wavelet threshold filtering can suppress Gaussian noise, it is not effective in processing non-stationary environmental interference, which can easily lead to missed detection or false alarm;

[0070] The time-current signal is transformed into The signal is decomposed into 32 frequency bands. When it is implemented:

[0071] Layers 1 and 2 decompose high-frequency noise (>1kHz), such as circuit switching noise;

[0072] The third and fourth layers decompose the medium-frequency noise (100Hz-1kHz), corresponding to the dynamic process of molecular adsorption;

[0073] The fifth layer decomposes low-frequency noise (<100 Hz), reflecting sensor baseline drift;

[0074] Specifically, this layering strategy is based on statistics from the ZCLS2000 detection data. The response signals of flammable and explosive liquids are mainly concentrated in the 50-500Hz frequency band, while the environmental noise energy is distributed throughout the entire frequency band. In particular, the low-frequency band (<100Hz) has a serious overlap with the signal frequency band, requiring multi-scale decomposition to achieve fine separation in the time-frequency domain.

[0075] By leveraging the multi-resolution analysis capabilities of wavelet packets, different modal signals are split from high to low frequency. The high-frequency band (>1kHz) captures transient interference such as circuit switching noise, the mid-frequency band (100Hz-1kHz) focuses on the dynamic process of molecular adsorption (such as ion migration and characteristic changes in Raman scattering), and the low-frequency band (<100Hz) reflects slow-varying noise such as sensor baseline drift to achieve multi-scale decomposition. Each frequency band sub-signal is quantized into a set of coefficients (i.e., detail coefficients).

[0076] Through 5-layer wavelet packet decomposition, the time current signal and frequency intensity signal It is split into 32 frequency band sub-signals, and then the quantization coefficients are extracted from the sub-signals to obtain the first detail coefficients (corresponding to the frequency band characteristics of the time current signal) and the second detail coefficient (corresponding to the frequency band characteristics of the frequency intensity signal);

[0077] The adaptive threshold control process is:

[0078] After decomposition, the first detail coefficient of each frequency band is obtained (time current signal) and the second detail coefficient (frequency intensity signal);

[0079] Specifically, the purpose of decomposition is to separate different modal signals according to their frequency characteristics, so as to facilitate targeted processing of slow-varying noise in time current signals (such as baseline drift, mainly in the low-frequency band) and shot noise in frequency intensity signals (distributed over the entire frequency band, especially near characteristic peaks, which require detailed processing);

[0080] Get the first detail coefficient and the second detail coefficient The first kurtosis of each frequency band coefficient and the second kurtosis ;

[0081] According to the first kurtosis and the second kurtosis , dynamically modify the kurtosis thresholds of different frequency bands. If any kurtosis is detected in a certain frequency band that is greater than or equal to 3.5 (i.e. or ), indicating the presence of dangerous goods signals, the threshold is multiplied by the correction factor , retaining weak features (such as weak characteristic peaks of low-concentration molecules in frequency intensity signals, or weak ion signals of trace explosive vapors in time current signals);

[0082] If the first kurtosis in a frequency band and the second kurtosis If both are less than 3.5, it is determined to be a noise-dominated frequency band. The threshold is compressed to enhance noise suppression (such as removing baseline drift noise of time current signals and shot noise of frequency intensity signals). The corrected threshold is:

[0083] ;

[0084] in, Dynamic adjustment, is the threshold The basic calculation factor of

[0085] After adaptive threshold adjustment, the frequency band coefficients of each mode are reconstructed and recombined into time domain signals (adjusted time current signals , frequency intensity signal ), return the signal to its original dimension and obtain the adjusted time current signal and frequency intensity signal , forming the best feature data Used for subsequent feature mining and pattern recognition;

[0086] Specifically, the reconstruction splits the signal characteristics into kurtosis discrimination noise through multi-scale decomposition, and finally restores the pure signal through dynamic threshold balanced noise reduction and feature reconstruction, realizing accurate preprocessing of dual-modal data of time-current signal and frequency-intensity signal. It not only solves the problem of suppressing thermal noise and environmental interference in security inspection scenarios, but also ensures the retention of weak characteristics of low-concentration dangerous goods.

[0087] S3: Based on the optimized DCAFEN, the spatiotemporal feature data is taken as input and the optimized spatiotemporal features are output;

[0088] The optimized DCAFEN (Deep Convolutional Attention Feature Extraction Network) architecture is designed as follows:

[0089] The optimized DCAFEN includes the input layer, multi-branch feature extraction layer, dual attention fusion layer, and output layer;

[0090] The input layer receives the best feature data , that is, the adjusted time current signal and frequency intensity signal ;

[0091] Multi-branch feature extraction layer:

[0092] Best feature data With the dual dimension of temporal dynamics (the order of occurrence of temporal current signals and the frequency distribution of frequency intensity signals), the Deep Convolutional Attention Feature Extraction Network (DCAFEN) processes in parallel through multi-branch input modules:

[0093] The time-current signal after adjustment in the best characteristic data , use 1D dilated convolution (kernel size 5, dilation rate 2) to capture long-distance dependent output time features ;

[0094] Specifically, the time features extracted from the best feature data , used to further distinguish the migration differences of different explosive ions (such as the time interval between the characteristic peaks of TNT and RDX ion signals);

[0095] Adjusted frequency intensity signal The frequency distribution characteristics (number and position of signal characteristic peaks) of the signal are generated by short-time Fourier transform (STFT) (window length 256, overlap rate 50%) to generate a time-frequency spectrum. , use 1×1 convolution to compress the number of channels to output frequency domain features ;

[0096] Specifically, the signal characteristic peak is the characteristic pulse or peak structure formed in the signal sequence after the dangerous goods molecules act on the detector. The characteristic peak of the signal refers to the current pulse formed by the movement of ions in the migration tube after the ionization of dangerous goods molecules. The frequency intensity signal after adjustment Here, the characteristic peak of the signal corresponds to the Raman characteristic peak, which is the peak intensity of scattered light generated by the vibration energy level transition of the dangerous goods molecules;

[0097] Dual attention fusion layer:

[0098] Time characteristics , calculate the energy share of each channel (output of different dilated convolution kernels) by global average pooling , the formula is:

[0099] ;

[0100] Where c is the channel index and t is the time index;

[0101] Frequency domain features Generate spatial weights using 3×3 convolution , highlighting the location of characteristic peaks in the time-frequency spectrum (e.g., a specific frequency-time window):

[0102] ;

[0103] in, is the activation function, is the convolution operation;

[0104] In order to avoid the gradient vanishing problem that is prone to occur in traditional serial fusion and to ensure the feature fusion effect and computational efficiency, a parallel weighted fusion method is used to integrate time and frequency domain features.

[0105] The channel attention weight , spatial attention weight and time characteristics , frequency domain characteristics Perform parallel fusion, the formula is: ;

[0106] Specifically, through parallel computing, different modal features (time dynamics, frequency domain distribution) can be fully integrated while avoiding the gradient vanishing effect that affects the model training effect;

[0107] For example, from the perspective of computational complexity, channel attention is based on global pooling, and the computational complexity is , spatial attention relies on 3×3 convolution, and the computational complexity is (T is the number of time windows, H is the number of frequency points). The computational complexity is lightweight, suitable for portable device computing resources, effectively suppressing environmental noise (such as airflow disturbances and noise generated by electromagnetic interference), and improving feature purity.

[0108] Output layer:

[0109] After completing the dual attention fusion layer fusion, the fusion features Perform compression adaptation processing;

[0110] Use 1×1 convolution kernel to fusion features Channel compression is performed to reduce the feature dimension from high dimension to 64 dimensions. The formula is expressed as: ;

[0111] Specifically, the 1×1 convolution kernel is used to reduce the model computation and storage overhead while retaining the key fusion features. The final output features That is, optimizing spatiotemporal features;

[0112] By deeply fusing and compressing the multimodal information of time-current signals and frequency-intensity signals obtained through ion mobility spectrometry (IMS) and Raman spectroscopy, this feature can be directly input into subsequent classification models for accurate identification of hazardous material types (such as distinguishing explosives), providing the core basis for rapid and accurate early warnings by detectors. The DCAFEN network on the ZC-ES3000 device can fully exploit multimodal signal characteristics and adapt to hardware computing resources, providing an effective technical path for precise and lightweight hazardous material detection.

[0113] S4: Based on the optimized spatiotemporal features, a spatiotemporal fingerprint database is constructed, and a dynamic logic gate judgment mechanism is designed to identify dangerous objects and output the hazard confidence results;

[0114] The process of building the spatiotemporal feature fingerprint library is as follows:

[0115] Extract the time core features and frequency core features from the optimized spatiotemporal features and encode them into fingerprints, which include time fingerprints and frequency fingerprints;

[0116] Time core features include;

[0117] Rising edge slope: Calculate the ratio of the current change from the start to the peak of the ion peak to the time difference, reflecting the rising speed of the ion peak. For example, the rising edge slope of TNT is about 0.5mA / ms.

[0118] Peak timestamp: records the moment when the current peak is reached in the time characteristic. The typical peak timestamp of TNT is 200ms.

[0119] Peak decay rate: Calculate the ratio of the current decay from the peak value to the time decay in the time characteristic. The peak decay rate of TNT is about 0.3mA / ms.

[0120] The above time core features are sequentially combined into a time fingerprint;

[0121] Frequency core features include:

[0122] Frequency coordinates: record the frequency value of frequency characteristics;

[0123] Peak intensity ratio: obtains the ratio of the intensity of a single characteristic peak in the frequency characteristics to the total intensity of all characteristic peaks, reflecting the relative intensity of the vibration mode;

[0124] Peak-to-peak ratio: Select the minimum frequency value in the frequency characteristics, obtain the ratio of the frequency difference between any two peaks to the minimum frequency, and quantify the relative distance between the peaks;

[0125] The above core features constitute the frequency fingerprint;

[0126] The acquired time fingerprint and frequency fingerprint are combined through knowledge graph technology to form a spatiotemporal feature fingerprint atlas library of dangerous goods. This is uniformly stored locally on the ZC-ES3000 device and used for subsequent detection to compare the real-time extracted features with the fingerprints in the atlas library.

[0127] The implementation process of the dynamic logic gate judgment mechanism is as follows:

[0128] Dynamic logic gates include basic gates, enhancement gates, and veto gates;

[0129] Specifically, the basic gate is the entry condition for classification, which must meet the key feature matching of time fingerprint and frequency fingerprint at the same time:

[0130] The basic gate implementation conditions are:

[0131] Time fingerprint condition: rising edge slope extracted in real time The rising slope of the spatiotemporal fingerprint library The difference is less than ,Right now , ensuring the matching of ion peak rising velocity characteristics;

[0132] Frequency fingerprint condition: the first characteristic peak frequency extracted in real time The first characteristic peak frequency of the spatiotemporal fingerprint library The difference is less than ,Right now , ensuring the core frequency matching of Raman characteristic peaks;

[0133] If both conditions are met at the same time, the detection signal is initially judged to enter the enhanced gate for further verification. If not, it is directly judged as a "non-target dangerous item";

[0134] The enhancement gate is used to strengthen the judgment result. Meeting any one of the conditions can increase the risk confidence:

[0135] Time fingerprint condition: real-time extraction of peak timestamps The peak timestamp of the spatiotemporal fingerprint database The interval indicates that the temporal characteristics of the ion peaks are highly matched;

[0136] Frequency fingerprint condition: the first peak intensity ratio extracted in real time The ratio of the first peak intensity to the spatiotemporal fingerprint library The difference is less than 0.1, that is , indicating that the Raman peak intensity distribution characteristics are consistent;

[0137] If both conditions are met, the hazard confidence level is increased by 25% by strengthening the gate condition;

[0138] The veto gate is used to eliminate the possibility of misjudgment. If the following characteristics are present, it will be directly judged as a "non-target dangerous item":

[0139] Temporal fingerprint condition: real-time extracted peak decay rate Peak decay rate of spatiotemporal fingerprint library The difference is greater than ,Right now , indicating that the ion peak attenuation characteristics are very different and it is not the target item;

[0140] Frequency fingerprint conditions: If more than one new unknown characteristic peak is detected, or the intensity of a single unknown peak exceeds 0.2, it means that the molecular vibration mode does not match the spatiotemporal characteristic fingerprint library and is not a target dangerous item;

[0141] Triggering any one of the veto gate conditions reduces the risk confidence by 50%. Triggering all conditions directly determines it as safe, and the risk confidence drops to 0%.

[0142] If the basic gate conditions are met, an initial 50% confidence level is assigned, indicating that the item is initially judged to be a target dangerous item. Each time an enhanced gate condition is passed, the confidence level increases by 20%, strengthening the judgment result. If any of the veto gate conditions is met, 50% confidence level is deducted. If all the veto gate conditions are met at the same time, the judgment is directly denied.

[0143] Finally, the device compares the confidence calculation results of the dangerous goods and outputs the dangerous goods category with the highest confidence. When the confidence of the dangerous goods is lower than the judgment threshold of 50%, the output is safe, otherwise the output is unsafe.

[0144] Example 2

[0145] like Figure 2 As shown in FIG, a highly sensitive dangerous goods detection system based on multimodal sensor data processing includes:

[0146] Data acquisition module: Captures electrical signals caused by dangerous goods molecules through multimodal sensors and generates multimodal data sequences;

[0147] Feature acquisition module: AMNSA is used to perform multi-scale decomposition of multimodal data sequences and then perform adaptive threshold adjustment to obtain optimal feature data;

[0148] Feature optimization module: takes spatiotemporal feature data as input based on the optimized DCAFEN and outputs optimized spatiotemporal features;

[0149] Hazard identification module: Build a spatiotemporal feature fingerprint library based on optimized spatiotemporal features, design a dynamic logic gate judgment mechanism to identify dangerous items, and output the hazard confidence results.

[0150] In the application, several formulas involved are calculated by taking their numerical values ​​after removing the dimensions, and the formulas are established by collecting a large amount of data and performing software simulation to obtain a formula for the most recent real situation. Some coefficients or weights in the formulas are set by technical personnel in this field according to actual conditions, so they will not be elaborated here.

[0151] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0152] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A highly sensitive dangerous goods detection method based on multimodal sensor data processing, characterized in that: include: S1: Capture the electrical signals caused by dangerous goods molecules through multimodal sensors to generate multimodal data sequences; S2: Adaptive multi-scale noise suppression (AMNSA) is used to perform multi-scale decomposition on the multimodal data sequence and then perform adaptive threshold adjustment to obtain the optimal feature data. The process of adaptive threshold adjustment is as follows: Based on the first detail coefficient and the second detail coefficient , get the first detail coefficient and the second detail coefficient The first kurtosis of each frequency band coefficient and the second kurtosis ; For any kurtosis greater than or equal to The threshold is multiplied by the correction factor ; First kurtosis and the second kurtosis The correction process for the threshold values ​​that are all less than 3.5 is: ； in, Dynamic adjustment, is the threshold The basic calculation factor of After adaptive threshold adjustment, the frequency band coefficients of each mode are reconstructed to obtain the adjusted time current signal and frequency intensity signal , forming the best feature data ; S3: Based on the optimized deep convolutional attention feature extraction network DCAFEN, spatiotemporal feature data is taken as input and optimized spatiotemporal features are output; The optimized deep convolutional attention feature extraction network DCAFEN includes an input layer, a multi-branch feature extraction layer, a dual attention fusion layer, and an output layer; The dual attention fusion layer implementation process is as follows: Time characteristics , calculate the energy proportion of each channel through global average pooling , the formula is: ； Where c is the channel index and t is the time index; Frequency domain features Generate spatial weights using 3×3 convolution ; The parallel weighted fusion method is used to integrate the time features and frequency domain features, and the channel attention weights are , spatial attention weight and time characteristics , frequency domain characteristics Perform parallel fusion to obtain the fusion features output by the dual attention fusion layer; S4: Based on the optimized spatiotemporal features, a spatiotemporal feature fingerprint library is constructed, and a dynamic logic gate judgment mechanism is designed to identify dangerous items and output the hazard confidence results.

2. The high-sensitivity dangerous goods detection method based on multimodal sensor data processing according to claim 1 is characterized in that: The multimodal data sequence includes a time current signal and a frequency intensity signal, and the multimodal sensor includes a portable explosive detector and a 785nm laser.

3. The high-sensitivity dangerous goods detection method based on multimodal sensor data processing according to claim 1 is characterized in that: The multi-scale decomposition process is as follows: Through the adaptive multi-scale noise suppression technology AMNSA, a 5-layer wavelet packet decomposition based on the db4 wavelet basis is used to decompose the time current signal into 32 frequency bands, and then the quantization coefficients are extracted from the decomposed sub-signals to obtain the first detail coefficients. and the second detail coefficient ; The first and second layers of wavelet packets decompose noise greater than 1kHz, the third and fourth layers of wavelet packets decompose noise from 100Hz to 1kHz, and the fifth layer of wavelet packets decomposes noise less than 100Hz.

4. The high-sensitivity dangerous goods detection method based on multimodal sensor data processing according to claim 1 is characterized in that: The multi-branch feature extraction layer implementation process is as follows: The time-current signal after adjustment in the best characteristic data , using 1D dilated convolution with kernel size 5 and dilation rate 2 to capture long-distance dependent output time features ; Adjusted frequency intensity signal in the best feature data The frequency distribution characteristics of the time spectrum are generated by short-time Fourier transform with a window length of 256 and an overlap rate of 50%. , use 1×1 convolution to compress the number of channels to output frequency domain features .

5. The high-sensitivity dangerous goods detection method based on multimodal sensor data processing according to claim 1 is characterized in that: The output layer implementation process is: Fusion features Perform compression adaptation processing and use 1×1 convolution kernel to fusion features Channel compression is performed to reduce the feature dimension from high dimension to 64 dimensions, and the optimized spatiotemporal features are output.

6. The high-sensitivity dangerous goods detection method based on multimodal sensor data processing according to claim 5 is characterized in that: The construction process of the spatiotemporal feature fingerprint library is as follows: Extract the time core features and frequency core features from the optimized spatiotemporal features and encode them as fingerprints; The fingerprint includes a time fingerprint and a frequency fingerprint; The time core features include rising edge slope, peak timestamp and peak decay rate; The frequency core features include frequency coordinates, peak intensity ratio and peak spacing ratio; The acquired time fingerprint and frequency fingerprint are combined through knowledge graph technology to form a spatiotemporal feature fingerprint library of dangerous goods.

7. The high-sensitivity dangerous goods detection method based on multimodal sensor data processing according to claim 6 is characterized in that: The implementation process of the dynamic logic gate determination mechanism is as follows: The dynamic logic gate determination mechanism includes a basic gate, an enhancement gate, and a veto gate; The basic gate time fingerprint condition is that the difference between the rising edge slope of the real-time extracted and the rising edge slope of the spatiotemporal feature fingerprint library is less than The basic gate frequency fingerprint condition is that the difference between the first characteristic peak frequency extracted in real time and the first characteristic peak frequency of the spatiotemporal characteristic fingerprint library is less than ; The enhanced gate time fingerprint condition is to extract the peak time stamp in real time and the peak time stamp of the spatiotemporal feature fingerprint database is Within the interval, the enhanced gate frequency fingerprint condition is that the difference between the first peak intensity ratio extracted in real time and the first peak intensity ratio of the spatiotemporal feature fingerprint library is less than 0.1; The rejection gate uses a time fingerprint condition when the difference between the peak decay rate extracted in real time and the peak decay rate of the spatiotemporal feature fingerprint library is greater than The rejection gate frequency fingerprint condition is that the number of newly detected unknown characteristic peaks exceeds 1, or the intensity ratio of a single unknown peak exceeds 0.

2.

8. A highly sensitive dangerous goods detection system based on multimodal sensor data processing, implemented according to the method according to any one of claims 1 to 7, characterized in that: include: Data acquisition module: Captures electrical signals caused by dangerous goods molecules through multimodal sensors and generates multimodal data sequences; Feature acquisition module: Adaptive multi-scale noise suppression technology AMNSA is used to perform multi-scale decomposition of multimodal data sequences and then perform adaptive threshold adjustment to obtain optimal feature data; Feature optimization module: Based on the optimized deep convolutional attention feature extraction network DCAFEN, it takes spatiotemporal feature data as input and outputs optimized spatiotemporal features; Hazard identification module: Build a spatiotemporal feature fingerprint library based on optimized spatiotemporal features, design a dynamic logic gate judgment mechanism to identify dangerous items, and output the hazard confidence results.

Images