A vehicle biological intrusion prevention intelligent monitoring method based on multi-sensor fusion

Abstract

The application discloses a kind of vehicle biological invasion intelligent monitoring methods based on multi-sensor fusion, it is related to vehicle-mounted intelligent safety monitoring and response technical field, including the multi-modal data of vehicle key area is synchronously collected by sensor, and pretreatment is carried out, obtain the multi-modal data after pretreatment;Feature extraction is carried out to the multi-modal data after pretreatment, extract frequency domain feature vector, voiceprint matching degree and vibration energy increment, and the confidence index of biological invasion is comprehensively calculated;According to confidence index stratified trigger response strategy, complete the vehicle biological invasion intelligent monitoring and response based on multi-modal perception.The application significantly improves the detection sensitivity and identification accuracy of small biological invasion in vehicle key area;Give consideration to low-power on-line operation and real-time protection needs, and reduce false alarm and miss risk by stratified response.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted intelligent safety monitoring and response technology, and in particular to an intelligent monitoring method for preventing biological intrusion into vehicles based on multi-sensor fusion. Background Technology

[0002] In recent years, with the rapid development of intelligent connected vehicles and in-vehicle safety technologies, multi-source information fusion and intelligent sensing technologies have received widespread attention in the field of vehicle safety protection. Early in-vehicle environmental monitoring relied heavily on single sensors, such as ultrasonic radar or infrared thermal imaging, to detect obstacles and environmental changes inside or outside the vehicle. However, single sensing channels are often limited by the inherent limitations of the sensors themselves: ultrasonic radar is susceptible to ranging errors due to complex reflective surfaces; infrared thermal imagers are not sensitive enough to ambient temperature and the thermal radiation response of obstructed objects; and vision-based monitoring systems are easily affected by changes in lighting and field-of-view obstruction.

[0003] However, existing technologies still have significant shortcomings in vehicle biological intrusion detection and response. First, many solutions only detect intrusions in the external environment of the vehicle, neglecting potential threats to critical areas such as the vehicle's interior or engine compartment. Furthermore, single radar or visual sensors often struggle to distinguish between small animals and non-biological disturbances, leading to high false alarm rates. Energy consumption is also a significant issue in continuous monitoring scenarios, failing to meet the requirements of low-power applications. Second, while multimodal fusion methods can improve detection accuracy, current work largely focuses on offline or cloud processing, which is insufficient to meet the real-time requirements and limited computing resources of the vehicle. In addition, existing solutions often rely on simple threshold triggers for risk assessment and response strategies, failing to comprehensively assess confidence levels and implement tiered responses for multi-source, multi-time-series data, posing a potential risk of overly simplistic or malfunctioning response actions. Summary of the Invention

[0004] In view of the problems existing in the current intelligent monitoring method for vehicle biological intrusion prevention based on multi-sensor fusion, this invention is proposed. Therefore, the problem to be solved by this invention is how to provide an intelligent monitoring method for vehicle biological intrusion prevention based on multi-sensor fusion.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] In a first aspect, the present invention provides a vehicle anti-biological intrusion intelligent monitoring method based on multi-sensor fusion, which includes synchronously collecting multimodal data of key areas of the vehicle through sensors and preprocessing it to obtain preprocessed multimodal data.

[0007] Feature extraction is performed on the preprocessed multimodal data, including frequency domain feature vectors, voiceprint matching degree, and vibration energy increment, and the confidence index of biological invasion is calculated comprehensively.

[0008] Based on a layered trigger response strategy using the confidence index, intelligent monitoring and response to vehicle biological intrusion based on multimodal perception is achieved.

[0009] As a preferred embodiment of the intelligent monitoring method for preventing biological intrusion into vehicles based on multi-sensor fusion described in this invention, the sensors include a millimeter-wave radar sensor, a microphone array, and a vibration sensor; the multimodal data includes vital sign signals collected by the millimeter-wave radar sensor, voiceprint frame data collected by the microphone array, and vibration energy data collected by the vibration sensor.

[0010] As a preferred embodiment of the intelligent vehicle anti-biological intrusion monitoring method based on multi-sensor fusion described in this invention, the preprocessing includes:

[0011] The obtained vital signs signals, voiceprint frame data, and vibration energy data are first aligned using a unified sampling clock interpolation, and then normalized by mean, as shown below:

[0012]

[0013] in: Let x(t) be the normalized data at time t, and let μ be the original data at time t. x σ is the mean of the original data. x This represents the standard deviation of the original data.

[0014] As a preferred embodiment of the intelligent vehicle anti-biological intrusion monitoring method based on multi-sensor fusion described in this invention, the feature extraction of the preprocessed multimodal data includes:

[0015] Frequency domain features were extracted from the preprocessed vital signs signals. Fast Fourier Transform was applied to each frame of vital signs signals to obtain the spectrum, and the peak amplitude of respiration and the peak amplitude of heartbeat were extracted.

[0016] The duration for which the peak amplitude of respiratory function exceeds the respiratory detection threshold or the peak amplitude of heart rate exceeds the heart rate detection threshold is recorded as the duration of vital signs.

[0017] To calculate the voiceprint matching degree, the similarity between the preprocessed voiceprint frame data and the pre-trained voiceprint template is calculated, and the maximum value is taken as the voiceprint matching degree.

[0018] Calculate the vibration energy increment based on the vibration energy data;

[0019] The determination of whether a biological invasion has occurred is based on a three-dimensional matrix. A biological invasion is determined only when all conditions are met simultaneously, as shown below:

[0020] Tlife >15s

[0021] M voice >0.85

[0022] ΔE vib >3dB

[0023] Wherein: T life For the duration of vital signs, M voice For voiceprint matching degree, ΔE vib This represents the increase in vibrational energy.

[0024] As a preferred embodiment of the intelligent vehicle anti-biological intrusion monitoring method based on multi-sensor fusion described in this invention, the expression for the voiceprint matching degree is:

[0025]

[0026] Wherein: S i Let ||·|| be the similarity between the voiceprint and the i-th type of voiceprint template, and ||·|| be the norm. Normalized voiceprint signal, M voice Voiceprint matching score;

[0027] The expression for the vibration energy increment is:

[0028] ΔE vib =E vib -E base

[0029] Where: ΔE vib E represents the increase in vibrational energy. base This is the static reference vibration energy.

[0030] As a preferred embodiment of the intelligent vehicle biological intrusion prevention monitoring method based on multi-sensor fusion described in this invention, the expression for the comprehensive calculation of the confidence index of biological intrusion is:

[0031]

[0032] Where: C bio ω1, ω2, and ω3 are confidence indices, and T is a weighted index. life For the duration of vital signs, T max M is the maximum duration of vital signs. voice For voiceprint matching degree, ΔE vib E represents the increase in vibrational energy. max This represents the maximum vibrational energy.

[0033] As a preferred embodiment of the intelligent vehicle anti-biological intrusion monitoring method based on multi-sensor fusion described in this invention, the hierarchical triggering response strategy based on confidence index includes:

[0034] Upon receiving a confidence index for biological invasion, the system determines the range of the confidence index. If the confidence index falls between the first and second limits, a local Level 1 response is initiated; otherwise, a local Level 2 response is initiated.

[0035] Local Level 1 Response: Sends a lighting control command to activate the hazard light mode, which flashes alternately at a predetermined frequency for a predetermined time. During the hazard light mode, the left and right turn signals flash alternately on and off. After the predetermined time, a cancel flashing signal is sent to restore normal lighting status.

[0036] The local level 2 response activates the ultrasonic drive-away device, which emits ultrasonic waves at a predetermined frequency. Simultaneously, it wakes up the vehicle's infrared camera, starts infrared video recording, acquires the current latitude and longitude coordinates, records the time of the event, and encapsulates it with the confidence index into an alarm message. The alarm message includes the time of the event, the vehicle's current location, the confidence index, and video data. After encapsulation, the alarm message is pushed to the remote monitoring platform in real time.

[0037] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a vehicle anti-biological intrusion intelligent monitoring method based on multi-sensor fusion.

[0038] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements the steps of a vehicle anti-biological intrusion intelligent monitoring method based on multi-sensor fusion.

[0039] The beneficial effects of this invention are as follows: This method significantly improves the detection sensitivity and identification accuracy of microbial intrusions in key areas of vehicles; it takes into account the requirements of low-power online operation and real-time protection, and reduces the risk of false alarms and missed alarms through hierarchical response, realizing quantifiable, traceable and highly adaptive intelligent monitoring and response to vehicle biological intrusions. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a vehicle anti-biological intrusion intelligent monitoring method based on multi-sensor fusion. Detailed Implementation

[0042] To make the above-mentioned objects, features, and advantages of the present invention more readily understood, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0043] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0044] Secondly, the term "one embodiment" or "example" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the invention. An embodiment appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment that selectively excludes other embodiments.

[0045] Reference Figure 1 This is the first embodiment of the present invention, which provides a vehicle anti-biological intrusion intelligent monitoring method based on multi-sensor fusion, including:

[0046] S1: Synchronously collect multimodal data of key areas of the vehicle through sensors, and preprocess the data to obtain preprocessed multimodal data;

[0047] Specifically, when the vehicle is turned off and engaged in parking brake mode, the onboard control unit (ECU) immediately sends a data acquisition start signal to each sensor module and distributes a unified clock synchronization flag via the vehicle's CAN bus to ensure that all subsequent data acquisition is accompanied by a precise timestamp. After receiving the start command, each module waits for the synchronization clock to begin the formal data acquisition process.

[0048] The millimeter-wave radar module extracts minute phase / frequency changes in respiration and heartbeat to construct vital sign waveforms, while a microphone array collects voiceprint frame data. Vibration sensors are deployed at key support points of the vehicle chassis to record acceleration data in real time and calculate vibration energy within each time window to reflect the low-frequency vibration intensity of the chassis or body structure, expressed as:

[0049]

[0050] Among them: E vib Vibrational energy within each time window, where t0 is the start time and T w Let a(t) be the acceleration data at time t, representing the time window.

[0051] Data preprocessing involves first aligning the obtained vital signs signals, voiceprint frames, and vibration energy according to a unified sampling clock interpolation, and then normalizing the data from each channel, as shown below:

[0052]

[0053] in: Let x(t) be the normalized data at time t, and let μ be the original data at time t. x σ is the mean of the original data. x This represents the standard deviation of the original data.

[0054] All sensor modules transmit raw signals and pre-processed features to the vehicle's main control unit via SPI or CAN bus in a unified timing sequence. The main control unit aligns these multi-source heterogeneous data streams based on a synchronous clock, and finally outputs a complete set of multimodal raw data packets, providing reliable input for subsequent feature fusion and intrusion detection.

[0055] S2: Perform feature extraction on the preprocessed multimodal data, extract frequency domain feature vectors, voiceprint matching degree and vibration energy increment, and comprehensively calculate the confidence index of biological invasion;

[0056] Specifically, frequency domain features are extracted from the preprocessed vital signs signals, and the spectrum is obtained by applying Fast Fourier Transform (FFT) to each frame of vital signs signals, and the peak amplitude of respiration and peak amplitude of heartbeat are extracted.

[0057] The duration for which the peak amplitude of respiratory function exceeds the respiratory detection threshold or the peak amplitude of heart rate exceeds the heart rate detection threshold is recorded as the duration of vital signs.

[0058] Voiceprint matching score is calculated by comparing the normalized spectrum of each voiceprint frame with the 12 pre-trained voiceprint templates, and taking the maximum value as the voiceprint matching score, expressed as:

[0059]

[0060] Wherein: S i Let ||·|| be the similarity between the voiceprint and the i-th type of voiceprint template, and ||·|| be the norm. Normalized voiceprint signal, M voice Voiceprint matching score;

[0061] The vibration energy increment is calculated based on the vibration energy and expressed as follows:

[0062] ΔE vib =E vib -E base

[0063] Where: ΔE vib E represents the increase in vibrational energy. base The energy of the static reference vibration;

[0064] The determination of whether a biological invasion has occurred is based on a three-dimensional matrix. A biological invasion is determined only if all three of the following conditions are met simultaneously, as shown below:

[0065] T life >15s

[0066] M voice >0.85

[0067] ΔE vib >3dB

[0068] Wherein: T life For the duration of vital signs, M voice For voiceprint matching degree, ΔE vib This represents the increase in vibrational energy.

[0069] The confidence index is calculated and expressed as:

[0070]

[0071] Where: C bio ω1, ω2, and ω3 are confidence indices, and T is a weighted index. life For the duration of vital signs, T max M is the maximum duration of vital signs. voice For voiceprint matching degree, ΔE vib E represents the increase in vibrational energy. max The maximum vibrational energy;

[0072] S3: Based on the confidence index, a hierarchical trigger response strategy is implemented to complete intelligent monitoring and response to vehicle biological intrusion based on multimodal perception.

[0073] Specifically, a tiered strategy is executed based on the output confidence index of biological intrusion: after the vehicle control unit (ECU) receives the confidence index of biological intrusion, it determines which preset range the value falls into: if the confidence level is between the first limit and the second limit, a local level 1 response is executed; if the confidence level exceeds the second limit, a local level 2 response is executed.

[0074] In a local Level 1 response, the main control unit sends a lighting control command to the vehicle control module to activate the hazard light mode. The hazard lights flash alternately three times per second for a predetermined time. During this period, the left and right turn signals flash alternately on and off. After the predetermined time, a cancel signal is sent to restore normal lighting status.

[0075] The local level 2 response activates the ultrasonic drive-away device. The main control unit drives the ultrasonic transducer to emit ultrasonic waves at a predetermined frequency. Simultaneously, the infrared camera integrated into the vehicle body is activated via the vehicle's communication network, initiating a five-second infrared video recording. The recorded data is temporarily stored in the vehicle's storage unit. After recording, the current latitude and longitude coordinates are obtained from the vehicle's GPS module, the event time is recorded, and these are encapsulated together with a confidence index into an alarm message. The alarm message includes the event time, the vehicle's current location, the confidence level, and the video data. After encapsulation, the alarm message is pushed in real-time to a remote monitoring platform via the mobile communication module, allowing the vehicle owner to be promptly informed and take further action.

[0076] This embodiment also provides a computer device applicable to a vehicle anti-biological intrusion intelligent monitoring method based on multi-sensor fusion, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement all or part of the steps of the method described in the above embodiments of the present invention.

[0077] This embodiment also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, it performs the method in any optional implementation of the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0078] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0079] In summary, this method significantly improves the detection sensitivity and identification accuracy of microbial intrusions in critical areas of vehicles; it takes into account the requirements of low-power online operation and real-time protection, and reduces the risk of false alarms and missed alarms through hierarchical response, realizing quantifiable, traceable and highly adaptive intelligent monitoring and response to vehicle biological intrusions.

[0080] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-sensor fusion-based intelligent monitoring method for preventing biological intrusion of a vehicle, characterized in that: The application relates to a vehicle biological intrusion intelligent monitoring method based on multi-sensor fusion. The multi-modal data of a key area of a vehicle is synchronously collected by a sensor, and preprocessed to obtain preprocessed multi-modal data; Feature extraction is performed on the preprocessed multi-modal data to extract a frequency domain feature vector, a voiceprint matching degree and a vibration energy increment, and a confidence index of biological intrusion is comprehensively calculated; The feature extraction on the preprocessed multi-modal data comprises the following steps: Frequency domain feature extraction is performed on the preprocessed vital sign signal, and a fast Fourier transform is applied to each frame of the vital sign signal to obtain a frequency spectrum, and a respiratory peak amplitude and a heartbeat peak amplitude are extracted; A duration that the respiratory peak amplitude exceeds a respiratory detection threshold or the heartbeat peak amplitude exceeds a heartbeat detection threshold is continuously detected and recorded as a vital sign duration; Voiceprint matching degree calculation is performed, and similarity is calculated between preprocessed voiceprint frame data and a pre-trained voiceprint template, and a maximum value is taken as the voiceprint matching degree; A vibration energy increment is calculated according to vibration energy data; Whether it is biological intrusion is determined according to a three-dimensional matrix, and only when the conditions are met simultaneously, is the biological intrusion determined, and is expressed as: wherein: is a duration of a vital sign, is a voiceprint matching degree, is a vibration energy increment; The expression of the comprehensive calculation of the confidence index of biological intrusion is: wherein: is a confidence index, , and is a weight index, is a vital sign duration, is a vital sign maximum duration, is a voiceprint match degree, is a vibration energy increment, is a maximum vibration energy; According to the confidence index, a response strategy is triggered in layers to complete the vehicle biological intrusion intelligent monitoring and response based on multi-modal sensing.

2. The multi-sensor fusion based intelligent monitoring method for vehicle bio-invasion prevention according to claim 1, characterized in that: The sensor comprises a millimeter wave radar sensor, a microphone array and a vibration sensor; and the multi-modal data comprises vital sign signals collected by the millimeter wave radar sensor, voiceprint frame data collected by the microphone array and vibration energy data collected by the vibration sensor.

3. The multi-sensor fusion based intelligent monitoring method for vehicle bio-invasion prevention according to claim 2, characterized in that: The preprocessing comprises the following steps: The vital sign signals, the voiceprint frame data and the vibration energy data are first aligned according to a unified sampling clock, and mean value normalization processing is performed, and is expressed as: wherein: is the normalized data at time is the original data at time is the mean of the original data, is the standard deviation of the original data.​​ 4. The multi-sensor fusion based intelligent monitoring method for vehicle bio-invasion prevention according to claim 3, characterized in that: The expression of the voiceprint matching degree is: wherein: is a similarity of the voiceprint and the first voiceprint template, is a norm; is a normalized voiceprint signal, is a voiceprint matching degree; The expression of the vibration energy increment is: wherein: is the incremental vibration energy, is the static reference vibration energy.

5. The multi-sensor fusion based intelligent monitoring method for vehicle bio-intrusion prevention according to claim 4, characterized in that: The response strategy triggered according to the confidence index in layers comprises the following steps: After the confidence index of biological intrusion is received, it is judged that the confidence index is in a limit value interval, if the confidence index is between a first limit value and a second limit value, a local first-level response is executed, and if the confidence exceeds the second limit value, a local second-level response is executed; The local first-level response sends a light control instruction, starts a double-flash light mode, alternately flashes at a predetermined frequency and lasts for a predetermined time, and during the double-flash light mode, left and right turn signals are alternately flashed at a rhythm, after the predetermined time ends, a flashing cancellation signal is sent, and a normal light state is restored; The local second-level response activates an ultrasonic driving device to emit ultrasonic waves at a predetermined frequency, wakes up an infrared camera of the vehicle body, starts infrared video recording, acquires current latitude and longitude coordinates, records an event occurrence time, and encapsulates the event occurrence time, the current position of the vehicle, the confidence index and video data into an alarm message, the alarm message comprises the event occurrence time, the current position of the vehicle, the confidence index and the video data, after the encapsulation is completed, the alarm message is pushed to a remote monitoring platform in real time. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the vehicle biological intrusion intelligent monitoring method based on multi-sensor fusion in any one of claims 1-5.

7. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by a processor to implement the steps of the vehicle biological intrusion prevention intelligent monitoring method based on multi-sensor fusion according to any one of claims 1-5.

Images