Intelligent security inspection method based on multi-element fusion

Abstract

The invention discloses an intelligent security inspection method based on multi-element fusion, and the method comprises the steps: collecting sound data in an environment through employing a microphone array, and calculating the position of a sound source based on the time difference, intensity difference and phase difference of the sound data; constructing a sound feature recognition model, and recognizing the type of a sounding subject in the sound source position and emotion deviation of a part of types of subjects; according to the identified emotional deviation of the person, analyzing the vocal meaning corresponding to the subject type by adopting an ASR technology, and judging whether the vocal meaning corresponds to risk feature information or not; based on the sound production meaning containing risk characteristic information, optical identification, gas component analysis, equipment pose, a point cloud map and wind direction meteorological comprehensive research and judgment analysis are combined, risk judgment output is carried out through a cloud edge collaborative algorithm, matched alarm information is output and played, and meanwhile lamplight flickering is carried out. The dangerous case is comprehensively researched and judged, and the risk identification accuracy and the dangerous case disposal efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security technology, and in particular to an intelligent security inspection method based on multi-factor fusion. Background Art

[0002] Currently, many security devices have been put into use, but the security systems built by most security devices mainly focus on deploying technical means such as visual recognition, thermal imaging monitoring and gas analysis in specific scenarios. However, this type of single-dimensional perception mode is essentially still at the "binary judgment" level, and is based on "whether a certain situation exists". That is, it can only verify the existence or non-existence of preset dangerous features, and lacks a comprehensive judgment of multi-factor integration, that is, a dynamic and correlated multi-factor analysis and judgment mechanism has not yet been formed. The technical architecture of this single-dimensional perception mode needs to be improved in accuracy and reliability in complex real-world inspection and identification scenarios. Existing security systems utilize a three-dimensional risk perception network formed through vision, hearing, and even touch. This perception gap directly restricts the security system's predictive capabilities. Key acoustic features such as the screams of people in distress, the unusual sounds of metal colliding, and the friction of speeding vehicles are not included in real-time monitoring, resulting in insufficient pre-judgment of dangerous situations. Fire monitoring relies solely on temperature thresholds to trigger alarms, but fails to capture changes in smoke particle concentration and unusual sounds in the early stages of combustion. Protection systems often only initiate tracking after an intrusion has occurred, missing the golden window for early warning through unusual sounds. Gas leak monitoring overly relies on single concentration value comparisons and fails to establish a spatiotemporal model of the leak source's diffusion path. Current alarm handling mechanisms remain at the rudimentary "trigger-alert" stage and lack intelligent decision-making support based on scenario semantics. When toxic gases spread, the system can only issue general alarms. While there are audio, light, and other alerts, there are no clearer alarms based on the specific circumstances of the dangerous situation. This leads to inadequate response methods when dangerous situations arise. Summary of the Invention

[0003] The purpose of the present invention is to provide an intelligent security inspection method based on multi-factor fusion to solve the problems raised in the above background technology.

[0004] The specific technical solution provided by the present invention is as follows: an intelligent security inspection method based on multi-factor fusion, comprising the following operating steps: Step S1: using a microphone array to collect sound data in the environment, and calculating the sound source position based on the time difference, intensity difference and phase difference of the sound data; Preferably, a microphone array is used to collect sound data in a security inspection environment, and the sound source position is determined based on the time difference, intensity difference, and phase difference of the sound data.

[0005] Step S2: Constructing a sound feature recognition model to identify the type of the speaker at the sound source location and the emotional bias of some types of speakers; Preferably, the types of the sound-making subject include: human, animal, vehicle, and others, and the type of subject with emotional bias is human; Preferably, identifying the type of the sound source in the sound source position includes: resampling the pre-processed original sound data in the sound data set to a set frequency; segmenting the audio frame to obtain an audio frame sequence, performing Fourier transform on the audio frame sequence and converting it into a spectrogram; processing the spectrogram using a Mel filter bank to obtain a Mel spectrogram, and performing normalization processing on the Mel spectrogram of the entire sound data set by calculating the average value and standard deviation; then randomly performing time warping and time-frequency masking on the sound data to obtain a new Mel spectrogram, and using the new Mel spectrogram to form a sound classification data set; Preferably, a sound feature recognition model is constructed, and the sound feature recognition model adopts a multi-layer perceptron, and the multi-layer perceptron includes an input layer, a hidden layer and an output layer; a sound classification data set composed of a new Mel-spectrogram is input into the input layer, and each node in the input layer corresponds to a feature of the input sound data, and the number of nodes is the type dimension of the input sound data, and the sound data type dimension includes four type dimensions of people, animals, vehicles, and others; a combination of hidden layer learning weights and activation functions is used to extract higher-level abstract features of the input sound data, and the identified features are matched and classified with the features of the four type dimensions originally defined in the system, and each node in the hidden layer is connected to all nodes in the input layer; the matching classification results of the sound data are output through the output layer, and the matching classification results are the type of the matched sound-making subject, and each output node corresponds to a type.

[0006] Step S3: Based on the emotional bias of the identified subject, ASR technology is used to analyze the meaning of the speech corresponding to the subject type, and determine whether the speech meaning corresponds to risk characteristic information; Preferably, according to the emotional bias corresponding to the output voice subject type, the corresponding ASR processing parameters are selected from the system preset configuration library to adjust the noise reduction intensity and frame length; at the same time, the acoustic model matching the emotional bias is loaded from the model warehouse, and the acoustic model is set according to different emotional biases. The setting rules are: a general model is used for calm and joyful emotions, and a special model is used for fear and anger emotions; the characteristics of the sound data changes caused by emotional bias are extracted, and the attention mechanism is used to focus on the emotion-sensitive frequency band. The language model is used simultaneously to generate multiple candidate hypotheses, and the acoustic model and language model adapted to the emotional bias are combined. The optimal one is screened through the re-scoring model related to the emotional bias. Path, give priority to retaining hypotheses containing emotionally biased keywords; predefine the risk feature vocabulary corresponding to the risk feature information of the voice meaning, as well as the severity corresponding to each risk word. According to the emotional intensity defined by the severity, use the BERT-based model to perform fine-grained sentiment analysis, and fuse the acoustic features with the defined risk feature vocabulary content to identify threatening semantics; finally, calculate the comprehensive risk score based on the emotional intensity and the severity of the risk word. Use the comprehensive risk score and the set threshold size to determine whether the voice meaning corresponding to the emotional bias has corresponding risk feature information. If the judgment result is yes, output the voice meaning of the emotional bias and the corresponding risk feature information.

[0007] Step S4: Based on the meaning of the voice containing risk feature information, combined with optical recognition, gas composition analysis, equipment posture, point cloud map and wind direction and meteorological comprehensive analysis, the cloud-edge collaborative algorithm is used to output risk judgment and matching alarm information; Preferably, the risk judgment output through the cloud-edge collaborative algorithm includes: Step S41: identifying requests for help, injuries, animal presence, rapid approach of vehicles, explosions, collisions, and locating the sound source according to steps S1-S3 based on the sound signal; Step S42: Analyze optical and infrared multimodal biometrics and motion using optical recognition technology. Step S43: Analyze the types, concentrations, and changing trends of harmful gases using gas composition analysis technology; Step S44: Analyze the three-dimensional coordinates, orientation, and timestamp corresponding to the current sound source position by combining the posture estimation analysis technology; Step S45: Providing multiple path plans starting from the current sound source position in combination with the point cloud map; Step S46: Combined with the multi-parameter weather station, wind direction, wind speed, temperature and humidity, air pressure, precipitation and visibility data are collected to provide the influencing factors of wind direction, cloudy, sunny, rainy, cloudy and snowy weather; Step S47: Use edge algorithms to analyze risks and output risk summaries and disposal recommendations.

[0008] Preferably, a lightweight risk assessment model including a data fusion layer, a feature extraction layer, a risk assessment layer and a decision-making layer is deployed at the edge computing node to map sound events, optical targets, gas concentrations, postures, paths and meteorological data to a unified spatiotemporal coordinate system to construct a multimodal graph structure; cross-modal feature interactions are learned through a graph attention network; the risk level is output based on the XGBoost model, and a risk summary is generated; finally, the preset disposal process is triggered according to the risk level, and the disposal strategy is optimized through reinforcement learning to output a comprehensive information recommendation text including risk type, location, impact range, recommended measures and evidence chain.

[0009] Step S5: Play the matching alarm information through speech synthesis technology, flash the light and send the alarm information to the server; Step S6: Within the set security inspection cycle, a cyclically coordinated security inspection is performed based on steps S1 to S5 through edge computing.

[0010] Compared with the prior art, the present invention has the following beneficial effects: (1) By introducing a microphone array to collect sound waves in the environment and combining multiple factors such as optical recognition and positioning, gas composition analysis, and sound detection, a comprehensive assessment of dangerous situations is achieved, overcoming the shortcomings of traditional manual inspections that are easily affected by the environment and weather, and improving the reliability and timeliness of security inspections; (2) The multi-factor fusion method breaks through the limitations of a single sensor, especially in complex monitoring scenarios, and can effectively avoid the problems of monitoring blind spots and overlapping areas, thereby improving the flexibility and adaptability of the monitoring system; (3) Through the collaborative work of multiple sensors and technologies, it can maintain stable operation in various harsh environments (such as low illumination, haze, etc.), effectively improving the recognition and response speed of abnormal behavior, and timely discovering and handling potential safety hazards; (4) Based on the fusion analysis of multiple factors, more accurate alarm information is given through the algorithm, which significantly improves the accuracy of risk identification, realizes risk prediction under certain circumstances, effectively reduces the occurrence of false alarms and missed alarms, and adds important dimensions to the security benchmark elements through multi-dimensional information fusion, comprehensively improving the intelligent level of security inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flowchart of the steps of the intelligent security inspection method based on multi-factor fusion provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the operational layered architecture of the intelligent security inspection method provided by an embodiment of the present invention. DETAILED DESCRIPTION The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention are within the scope of protection of the present invention.

[0012] Example 1: like Figure 1 As shown, this embodiment describes an intelligent security inspection method based on multi-factor fusion, including: Step S1: using a microphone array to collect sound data in the environment, and calculating the sound source position based on the time difference, intensity difference and phase difference of the sound data; In this embodiment, three or more microphones are distributed at different locations in a collection environment, and sound data received by each microphone is collected to form a collected sound data set. All sound data in the sound data set are preprocessed including bandpass filtering, frame windowing, and noise suppression. Exemplarily, a time difference calculation is performed on the preprocessed sound data, the sound data received by each two microphones is converted into the frequency domain using the TDOA technology, the cross-power spectrum thereof is calculated, and PHAT weighting is applied, a time domain cross-correlation function is obtained by inverse Fourier transform, a peak position is determined in the cross-correlation function, and the time offset corresponding to the peak is the time difference between the two microphones receiving the same sound data, and a hyperbolic equation system based on the reference microphone is established using the obtained time difference and the geometric position relationship between the microphones, the nonlinear problem is converted into a linear equation system using the Chan algorithm, and an initial estimate is obtained by the least squares method, and the estimate is the initial estimated sound source position; The energy of the sound data is further calculated within the time window, that is, the sum of the squares of the sound data amplitudes is calculated. For each pair of microphones, the energy ratio of the signals they receive is calculated to obtain the intensity difference. Based on the law that the energy attenuation of sound wave propagation is inversely proportional to the propagation distance, the intensity difference corresponds to the ratio of the distances between the two microphones and the sound source. The intensity difference is converted into a distance ratio relationship and used as a constraint condition for the final sound source location. Finally, the sound data is divided into frames, and a Fourier transform is performed on each frame to obtain a spectrum. The phase value of each pair of microphones is extracted at a specific frequency point in the frequency domain, and the phase difference between the two microphones at the same frequency point is calculated. Based on the condition that the microphone spacing is less than half a wavelength, the true phase difference is calculated. The relationship between the phase difference and the time difference is used to calculate the time difference of the corresponding frequency points. The second WLS estimation is performed using this time difference and the above constraints to obtain the second estimated value. The Taylor sequence expansion method is used to further optimize and iterate the second estimated value until convergence to obtain the determined sound source position and direction.

[0013] Step S2: Constructing a sound feature recognition model to identify the type of the speaker at the sound source location and the emotional bias of some types of speakers; In this embodiment, the types of sound-making subjects include: people, animals, vehicles, and others, and the subject with emotional bias is people; the original sound data pre-processed in the sound dataset is resampled to a set frequency, and the audio frames are segmented to obtain an audio frame sequence, the audio frame sequence is Fourier transformed and converted into a spectrogram form, the spectrogram is processed using a Mel filter bank to obtain a Mel spectrogram, the Mel spectrogram of the entire sound dataset is normalized by calculating the mean and standard deviation, and then the sound data is randomly time-warped and time-frequency masked to obtain a new Mel spectrogram, thereby completing the establishment of the sound classification dataset; Exemplarily, a sound feature recognition model is constructed. The sound feature recognition model adopts a multi-layer perceptron, which is composed of multiple neurons, including an input layer, a hidden layer and an output layer. The new Mel-spectrogram is received through the input layer. Each node corresponds to a feature of the input sound data. The number of nodes in the input layer is the dimension of the input sound data, which in this embodiment includes four dimensions: people, animals, vehicles, and others. The hidden layer contains multiple nodes, each node is connected to all nodes in the previous layer, and a combination of hidden layer learning weights and activation functions is used to extract higher-level abstract features of the input sound data. The output layer is used to output the dimensional classification result of the final sound data, that is, the type of the sound-emitting subject in the identified sound source position is output, and each output node corresponds to a type.

[0014] Furthermore, after determining the type of the sound-making subject at the sound source position, when the type of the sound-making subject is a human, the emotional bias model is used to identify and judge the emotional bias type corresponding to the sound data of the sound-making subject. The emotional bias types defined in this embodiment include calmness, fear, joy, and anger, etc. Specifically, when the subject type is "human", the emotional bias type is identified and matched through the frequency corresponding to the sound data of the corresponding person, and the emotional bias corresponding to the sound-making subject is output according to the identification and matching result.

[0015] Step S3: Based on the emotional bias of the identified subject, ASR technology is used to analyze the meaning of the speech corresponding to the subject type, and determine whether the speech meaning corresponds to risk characteristic information; In this embodiment, according to the output emotional bias type, the corresponding ASR processing parameters are selected from the preset configuration library, and the noise reduction intensity and frame length are adjusted. That is, the sound data under the anger type retains high-frequency energy, and the sound data under the fear type forms short frames to capture rapid changes. The acoustic model matching the emotional bias is loaded from the model warehouse. The acoustic model is set according to different emotional biases. The setting rules are: a general model is used for calm and joyful emotions, and a special model is used for fear and anger emotions. The model structure includes extracting the sound change characteristics caused by adaptive emotions and an attention mechanism focusing on emotion-sensitive frequency bands. The acoustic model adapted to the emotional bias is combined with the language model, and multiple candidate hypotheses are generated using the language model. The optimal path is screened through the emotional bias-related re-scoring model, and hypotheses containing emotional bias keywords are retained first. Furthermore, a risk feature vocabulary of risk feature information corresponding to the predetermined voice meaning and the severity corresponding to each risk word are synchronously defined according to the severity; a BERT-based model is used for fine-grained sentiment analysis, and the acoustic features are integrated with the defined risk feature vocabulary content to identify threatening semantics; a comprehensive risk score is calculated based on the emotion intensity and the severity of the risk word, and the comprehensive risk score and the set threshold are used to determine whether the voice meaning corresponding to the emotional bias has corresponding risk feature information. When the judgment result is yes, the voice meaning of the emotional bias and the corresponding risk feature information are output.

[0016] Step S4: Based on the meaning of the voice containing risk feature information, combined with optical recognition, gas composition analysis, equipment posture, point cloud map and wind direction meteorological comprehensive analysis, risk judgment output is made through the cloud-edge collaborative algorithm, and the optimal alarm information is output. Step S41: Identify requests for help, injuries, animal presence, rapid approach of vehicles, explosions, collisions, etc. through sound signals according to the above steps S1-S3, and locate the sound source. Step S42: Combine optical recognition technology to analyze optical and infrared multimodal organisms and movements, specifically including: analyzing the presence of organisms, speed approaching key areas, acceleration per unit time, brightness changes, temperature changes (infrared), etc. For example, optical cameras and infrared thermal imagers are deployed to monitor key areas in real time. Optical data is used through the YOLOv8 target detection model to identify the presence of living organisms and track their movement trajectories. The gait analysis algorithm is combined to calculate the displacement change per unit time, and the speed and acceleration parameters are calculated. Infrared data is used to detect changes in biological surface temperature and intrusion into hidden areas. At the same time, the sound source positioning results are combined to verify the spatial correlation of the optical target, that is, the overlap between the location of the sound event and the coordinates of the optical target is analyzed to form a joint representation of the biological presence, movement state and spatial distribution.

[0017] Step S43: Combining gas composition analysis technology to analyze the types, concentrations and changing trends of harmful gases; For example, an electrochemical sensor array and a laser spectrometer are used to perform high-precision detection of harmful gases such as CO, H2S, and VOC in the environment. The detection data is preprocessed, and automatic baseline correction and outlier filtering are performed. A concentration trend prediction model is simultaneously constructed based on an LSTM neural network. Historical concentration sequences, meteorological conditions, and surrounding industrial activity data are input, and a concentration change curve for a set time period in the future is output. When the predicted value exceeds the safety threshold, the gas diffusion path simulation is triggered to generate a heat map of the risk area.

[0018] Step S44: Analyze the current three-dimensional coordinates, orientation, and timestamp by combining pose estimation and analysis technology; For example, IMU, GPS, UWB and visual SLAM data are integrated to build high-precision pose estimation technology. IMU is used to provide short-term, high-frequency attitude angle and acceleration data, GPS is used to output global coordinates, and UWB base stations are used to achieve centimeter-level indoor positioning. Visual SLAM corrects accumulated errors through feature point matching, uses extended Kalman filtering to fuse multi-source data, dynamically adjusts the weight of each sensor, and finally outputs a pose package containing three-dimensional coordinates, quaternion attitude and timestamp to provide a real-time position reference.

[0019] Step S45: Providing multiple path plans starting from the current location in combination with the point cloud map; For example, a high-precision point cloud map is constructed through lidar or depth camera, and obstacle information such as moving vehicles and falling objects is updated in real time. The Octomap algorithm is used to voxel compress the point cloud to reduce the computational load, and the initial path is generated using the current pose and target point coordinates. The path smoothness is then optimized using the elastic band algorithm. The DLite algorithm is introduced to achieve local replanning for dynamic risks to ensure that the path always bypasses high-risk areas. Finally, multiple alternative paths including the shortest path, the safest path, and the path with the best energy consumption and their risk assessment values ​​are output.

[0020] Step S46: Combined with the multi-parameter weather station, wind direction, wind speed, temperature and humidity, air pressure, precipitation and visibility data are collected to provide wind direction, cloudy, sunny, rainy, cloudy and snowy factors; Exemplarily, the collected meteorological data is imported into the edge computing node via RS485 or wireless transmission to generate a structured meteorological package containing a timestamp, element value, and confidence level; Step S47: Using edge algorithms to analyze risks, output risk summaries and disposal recommendations; In this embodiment, a lightweight risk assessment model consisting of a data fusion layer, a feature extraction layer, a risk assessment layer, and a decision-making layer is deployed at the edge computing node. The data fusion layer is used to map sound events, optical targets, gas concentrations, postures, paths, and meteorological data to a unified spatiotemporal coordinate system to construct a multimodal graph structure, in which nodes are entities and edges are associations. The feature extraction layer uses a graph attention network to learn cross-modal feature interactions. The risk assessment layer outputs the risk level based on the XGBoost model and generates a risk summary. Finally, the decision-making layer triggers the preset disposal process according to the risk level, and optimizes the disposal strategy through reinforcement learning, ultimately outputting a comprehensive information text containing the risk type, location, scope of impact, recommended measures, and chain of evidence.

[0021] Specifically, the operation method of the lightweight risk assessment model includes: Set three analysis indicators, including sound analysis, optical analysis and gas composition analysis; Sound analysis mainly uses the PyTorch+Conv1d model. Through the above steps, it determines whether there are human calls for help, screams, or groans of victims; whether there are animal calls, whether there are vehicles approaching and the distance reaches the threshold; and whether there are high-intensity sounds suspected to be explosions. Optical analysis uses the above steps to detect and judge whether there are animals and human bodies, whether animals are continuously approaching human bodies without obvious deceleration, whether animals are continuously approaching and accelerating, whether there is a sharp increase in brightness in the area, and whether there is a temperature above 80 degrees. Gas composition analysis is to calculate the presence of harmful gases and the concentration of harmful gases, as well as the intensity change trend of harmful gas concentrations at different locations through the above steps. When at least one of the above three indicators is triggered, the following risk indicators are calculated: Human distress indicator: The presence of human calls for help, screams, or groans from victims indicates distress. The distress indicator is 50%. The risk increases by 10% for every additional second of duration. Animal attack risk indicators: The risk of an animal being present is 20%, the risk of an animal continuously approaching increases by 10%, the risk of an animal accelerating to approach increases by 20%, the risk of an animal approaching less than 3 meters increases by 20%, and the risk of an animal approaching and making sounds increases by 20%. Vehicle risk indicator: Uses sound and optics to determine if a vehicle is approaching from behind. Predicts collision time based on current speed. If the time is less than 5 seconds, the risk is 50%. Every second of reduction increases the risk by 10%. The optimal evacuation direction is sideways, with left and right determined based on the map and current optical mapping. Hazardous gas analysis: If harmful gases exist and reach a certain threshold, set as 10% for mild, 50% for moderate, 70% for severe, and 100% for very severe, the location of the harmful gas source and the best safe direction can be inferred based on the relationship between the position change over a period of time and the concentration of the harmful gas. The above three analysis combinations distinguish between two types of risks: those requiring rescue and those requiring evacuation; at the same time, the risk sub-items are reported to the center, and the best evacuation direction is given based on the map, and on-site warning content is generated and synthesized voice. If there is rain, snow, strong wind, etc., regardless of whether the aforementioned risks exist, the "Beware of slippery roads" alarm is played continuously at a fixed frequency. Step S5: Play the alarm information through speech synthesis technology, flash the light and send the alarm information to the server. Step S6: During the set security inspection cycle, a cyclically coordinated security inspection is performed based on steps S1 to S5 through edge computing, where the edge-side control program is deployed on the main control PC in the device.

[0022] It should be noted that, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0023] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent security inspection method based on multi-factor fusion, characterized by: The following steps are included: Step S1: using a microphone array to collect sound data in the environment, and calculating the sound source position based on the time difference, intensity difference and phase difference of the sound data; Step S2: Constructing a sound feature recognition model to identify the type of the speaker at the sound source location and the emotional bias of some types of speakers; The types of the sound-making subject include: people, animals, vehicles, and others, and the type of subject with emotional bias is people; Step S3: Based on the emotional bias of the identified subject, ASR technology is used to analyze the meaning of the speech corresponding to the subject type, and determine whether the speech meaning corresponds to risk characteristic information; Step S4: Based on the meaning of the voice containing risk feature information, combined with optical recognition, gas composition analysis, equipment posture, point cloud map and wind direction and meteorological comprehensive analysis, the cloud-edge collaborative algorithm is used to output risk judgment and matching alarm information; Step S5: Play the matching alarm information through speech synthesis technology, flash the light and send the alarm information to the server; Step S6: Within the set security inspection cycle, a cyclically coordinated security inspection is performed based on steps S1 to S5 through edge computing.

2. The intelligent security inspection method based on multi-factor fusion according to claim 1 is characterized by: Calculating the sound source position includes: using a microphone array to collect sound data in a security inspection environment, and determining the sound source position based on the time difference, intensity difference, and phase difference of the sound data.

3. The intelligent security inspection method based on multi-factor fusion according to claim 2 is characterized by: Identifying the sound source location includes: Resample the pre-processed original sound data in the sound dataset to the set frequency; Segment the audio frames to obtain an audio frame sequence, perform Fourier transform on the audio frame sequence, and convert it into a spectrum graph form; The mel-spectrogram is processed by the mel filter bank to obtain the mel-spectrogram, and the mel-spectrogram of the entire sound dataset is normalized by calculating the mean and standard deviation; Then, the sound data is randomly time-distorted and time-frequency masked to obtain a new Mel-spectrogram, which is used to form a sound classification dataset. Constructing a sound feature recognition model, wherein the sound feature recognition model adopts a multi-layer perceptron, and the multi-layer perceptron includes an input layer, a hidden layer, and an output layer; Input the new sound classification dataset composed of mel-spectrograms into the input layer. Each node in the input layer corresponds to a feature of the input sound data. The number of nodes is the type dimension of the input sound data. The sound data type dimension includes four type dimensions: human, animal, vehicle, and other. Using a combination of hidden layer learning weights and activation functions, we extract higher-level abstract features of the input sound data and match and classify the identified features with the features of the four type dimensions originally defined in the system. Each node in the hidden layer is connected to all nodes in the input layer. The matching classification results of the sound data are output through the output layer. The matching classification results are the types of the matching sound-making subjects, and each output node corresponds to a type.

4. The intelligent security inspection method based on multi-factor fusion according to claim 3 is characterized by: The emotional biases of some types of subjects in identifying the sound source location include: After determining the type of the sound source, if the sound source is a person; Types of defined emotional biases include calm, fear, joy, and anger; Identify and match the emotional bias type of the corresponding person through the frequency corresponding to the voice data of the corresponding person; The emotional bias corresponding to the speaker is output based on the recognition and matching results.

5. The intelligent security inspection method based on multi-factor fusion according to claim 4 is characterized by: Determining whether the meaning of the utterance corresponds to the risk characteristic information includes: According to the emotional bias corresponding to the output voice subject type, the corresponding ASR processing parameters are selected from the system preset configuration library to adjust the noise reduction intensity and frame length; At the same time, the acoustic model that matches the emotional bias is loaded from the model warehouse. The acoustic model is set according to different emotional biases. The setting rule is: use a general model for calm and joyful emotions, and use a dedicated model for fear and anger emotions; Extract the characteristics of sound data changes caused by emotional bias, use the attention mechanism to focus on emotion-sensitive frequency bands, and simultaneously use the language model to generate multiple candidate hypotheses. Combine the acoustic model and language model adapted for emotion bias, and use the emotion-related re-scoring model to screen the optimal path, giving priority to hypotheses containing emotion-biased keywords. A risk signature vocabulary is predefined, which maps the meaning of each utterance to risk signature information, as well as the severity level associated with each risk word. Based on the emotional intensity defined by the severity level, a BERT-based model is used for fine-grained sentiment analysis. The acoustic features are then integrated with the defined risk signature vocabulary to identify threatening semantics. Based on the intensity of emotions and the severity of risk words, a comprehensive risk score is calculated. The comprehensive risk score and the set threshold are used to determine whether the vocal meaning corresponding to the emotional bias has corresponding risk characteristic information. When the judgment result is yes, the vocal meaning of the emotional bias and the corresponding risk characteristic information are output.

6. The intelligent security inspection method based on multi-factor fusion according to claim 5 is characterized by: The risk judgment outputs through the cloud-edge collaborative algorithm include: Step S41: identifying requests for help, injuries, animal presence, rapid approach of vehicles, explosions, collisions, and locating the sound source according to steps S1-S3 based on the sound signal; Step S42: Analyze optical and infrared multimodal biometrics and motion using optical recognition technology. Step S43: Analyze the types, concentrations, and changing trends of harmful gases using gas composition analysis technology; Step S44: Analyze the three-dimensional coordinates, orientation, and timestamp corresponding to the current sound source position by combining the posture estimation analysis technology; Step S45: Providing multiple path plans starting from the current sound source position in combination with the point cloud map; Step S46: Combined with the multi-parameter weather station, wind direction, wind speed, temperature and humidity, air pressure, precipitation and visibility data are collected to provide the influencing factors of wind direction, cloudy, sunny, rainy, cloudy and snowy weather; Step S47: Use edge algorithms to analyze risks and output risk summaries and disposal recommendations.

7. The intelligent security inspection method based on multi-factor fusion according to claim 6 is characterized by: The step S47 specifically includes: Deploy a lightweight risk assessment model on edge computing nodes, which includes a data fusion layer, a feature extraction layer, a risk assessment layer, and a decision-making layer. The data fusion layer is used to map sound events, optical targets, gas concentrations, postures, paths, and meteorological data into a unified spatiotemporal coordinate system to construct a multimodal graph structure; The feature extraction layer is used to learn cross-modal feature interactions through a graph attention network; The risk assessment layer is used to output the risk level based on the XGBoost model and generate a risk summary; The decision-making layer is used to trigger the preset disposal process according to the risk level, and at the same time optimize the disposal strategy through reinforcement learning, and output a comprehensive information recommendation text including risk type, location, scope of impact, recommended measures and evidence chain.

8. The intelligent security inspection method based on multi-factor fusion according to claim 7 is characterized by: The lightweight risk assessment model includes: Set analysis indicators including sound analysis, optical analysis and gas composition analysis; When at least one of the above three indicators is triggered, all risk indicators are calculated: Combining the above three assessments, we can distinguish between two types of risks: those requiring rescue and those requiring evacuation. At the same time, the risk sub-items are reported to the center, and the best evacuation direction is given based on the map. On-site warning content is generated and the warning content is synthesized into voice output; At the same time, for specific risk scenarios, execution rules are set to continuously play alarm information at a fixed frequency regardless of whether the aforementioned risks exist.

9. The intelligent security inspection method based on multi-factor fusion according to claim 8, characterized in that: The sound analysis uses the PyTorch+Conv1d model to judge risk scenarios containing sound; The optical analysis uses infrared imaging devices and optical sensors to optically identify and judge the relevant detection environment; the gas component analysis uses gas sensors to detect and judge the concentration of gas components in the environment.

10. The intelligent security inspection method based on multi-factor fusion according to claim 9 is characterized in that: All risk indicators mentioned include: human distress indicator, animal attack risk indicator, vehicle risk indicator, and harmful gas analysis indicator.

Images