Lateral backward visual safety protection method based on multi-modal fusion

Abstract

The invention discloses a lateral backward visual safety protection method based on multi-modal fusion, particularly relates to the field of safety protection, and comprises the steps of protection time division, data acquisition, data analysis, comprehensive analysis and safety protection. The safety risk in the vehicle driving process can be accurately evaluated by carefully dividing the protection time and comprehensively collecting and analyzing the environment, the vehicle state, obstacles and sensor data, meanwhile, the potential danger can be timely found and early warned by utilizing the comprehensive abnormal index calculated by the comprehensive analysis model, and the safety of the vehicle is improved. According to the invention, timely protection measures are provided for a driver or an automatic control system, so that the safety of vehicle driving is remarkably improved, the economic loss and casualty risks caused by safety accidents are effectively reduced, and the system has wide application prospects and important social value.

Description

Technical Field

[0001] The present invention relates to the field of safety protection technology, and more specifically, to a side and rear visual safety protection method based on multimodal fusion. Background Art

[0002] With the rapid development of the automobile industry and the continuous advancement of intelligent technology, vehicle safety protection technology has become a hot research area today, especially in the side and rear safety protection of vehicles. How to effectively monitor and warn of potential dangers and ensure driving safety has become a key issue that the industry needs to solve urgently.

[0003] At present, most vehicle safety protection systems on the market rely on a single sensor or technical means, such as radar, camera or ultrasonic sensor. These systems usually collect environmental information around the vehicle, perform simple data processing and judgment, and then issue warnings to the driver or take corresponding control measures.

[0004] However, the existing technology still has some obvious shortcomings in practical applications. For example, a single sensor is easily interfered by environmental factors, resulting in false alarms or missed alarms; at the same time, the existing safety protection system often lacks comprehensive analysis and judgment capabilities, making it difficult to accurately assess the overall safety risks of the vehicle during driving. These problems have seriously affected the safety and reliability of vehicle driving. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a side and rear visual safety protection method based on multimodal fusion, and solves the problems raised in the above-mentioned background technology through the following scheme.

[0006] To achieve the above object, the present invention provides the following technical solution: a side and rear visual safety protection method based on multi-modal fusion, comprising: Step 1: Protection time division: used to determine the safety protection time of the target vehicle as the target time area, divide the target time area into various sub-time areas by equal time division, and mark them as 1, 2...n in sequence; Step 2: Data collection: used to collect environmental data, vehicle status data, obstacle data and sensor data in each sub-time zone, and transmit the collected data to step 3; Step 3: Data analysis: including an environmental data analysis method, a vehicle status data analysis method, an obstacle data analysis method, and a sensor data analysis method, for analyzing the data transmitted in step 2, and transmitting the analysis results to step 4; Step 4: Comprehensive analysis: The data transmitted in step 3 is comprehensively analyzed through comprehensive analysis, and the analysis results are transmitted to step 5; Step 5: Safety protection: The comprehensive abnormality index of the target vehicle is judged by establishing a preset value of the comprehensive abnormality index, and a control instruction is issued according to the judgment result.

[0007] Preferably, the environmental data include current road visibility, ambient light intensity, road slip coefficient and ambient temperature, which are marked as Ev, El, Er and Ea respectively; the vehicle status data include vehicle real-time speed, vehicle steering angle, brake pedal opening and accelerator pedal opening, which are marked as Vs, Va, Vb and Vt respectively; the obstacle data include obstacle relative distance, obstacle relative speed, obstacle relative azimuth and obstacle size area, which are marked as Or, Ov, Oa and Od respectively; the sensor data include millimeter wave radar detection intensity, ultrasonic sensor echo time, camera image clarity and infrared sensor temperature difference, which are marked as Dr, Du, Dc and Dt respectively.

[0008] Preferably, the current road visibility is measured by emitting a laser beam through a laser visibility sensor to measure the scattering and attenuation of the laser by the atmosphere, the intensity of the scattered light is inversely proportional to the visibility, the ambient light intensity is measured by using a photodiode to convert light energy into an electrical signal, and the ambient light intensity is measured by a light intensity sensor to automatically adjust the vehicle lighting system and the exposure parameters of the auxiliary driving camera, the road slipperiness coefficient is calculated by using a special friction coefficient sensor by measuring the mechanical friction between the tire and the road surface, and the ambient temperature is measured using the resistance characteristics of an NTC thermistor that changes with temperature, which affects the engine performance control and air-conditioning system regulation.

[0009] Preferably, the real-time speed of the vehicle is calculated by detecting the pulse signal generated by the rotation of the gear through the Hall sensor installed on the wheel. The vehicle steering angle uses an angle encoder to detect the steering wheel rotation angle, which can be divided into incremental and absolute types. The brake pedal opening is measured by a linear potentiometer to measure the brake pedal displacement and converted into a percentage value. The accelerator pedal opening uses a non-contact potentiometer to measure the accelerator pedal position for engine power output control.

[0010] Preferably, the relative distance of the obstacle represents that the millimeter wave radar transmits electromagnetic waves, and the distance is calculated by measuring the reflected wave time. The relative speed of the obstacle is based on the Doppler effect, and the relative speed is calculated by measuring the change in the reflected wave frequency. The relative azimuth of the obstacle uses a multi-channel radar array, and the target azimuth is determined by the phase difference of the signals received by adjacent antennas. The size and area of ​​the obstacle is acquired by collecting images with a camera, and the target contour is extracted and the area is calculated using a computer vision algorithm.

[0011] Preferably, the millimeter wave radar detection strength is achieved by measuring the power strength of the radar receiving signal, the ultrasonic sensor echo time is achieved by recording the time interval from ultrasonic emission to reception of the reflected wave, the camera image clarity is achieved by calculating the ratio of signal to noise in the image to evaluate the image quality, and the infrared sensor temperature difference is achieved by measuring the temperature difference between the target and the environment through the infrared sensor, which is used for nighttime pedestrian detection and special object recognition.

[0012] Preferably, the environmental data analysis method is used to establish an environmental data analysis model, import the environmental data transmitted in step 2 into the environmental data analysis model, and calculate the environmental assessment value of each sub-time zone, which is specifically expressed as: , E i represents the environmental assessment value of the ith sub-time zone, Ev i represents the current road visibility in the ith sub-time zone, El i represents the ambient light intensity of the ith sub-time zone, Er i Indicates the road slip coefficient of the ith sub-time zone, Ea i Represents the ambient temperature of the i-th sub-time zone.

[0013] Preferably, the vehicle state data analysis method is used to establish a vehicle state data analysis model, import the vehicle state data transmitted in step 2 into the vehicle state data analysis model, and calculate the vehicle state evaluation value of each sub-time zone, which is specifically expressed as: , V i represents the vehicle state evaluation value of the i-th sub-time zone, Vs i represents the real-time speed of the vehicle in the ith sub-time zone, Va i represents the vehicle steering angle in the ith sub-time zone, Vb i represents the brake pedal opening in the ith sub-time zone, Vt i Indicates the accelerator pedal opening degree in the i-th sub-time zone.

[0014] Preferably, the obstacle data analysis method is used to establish an obstacle data analysis model, import the obstacle data transmitted in step 2 into the obstacle data analysis model, and calculate the obstacle evaluation value of each sub-time zone, which is specifically expressed as: , O i represents the obstacle assessment value of the ith sub-time zone, Or i represents the relative distance of obstacles in the ith sub-time zone, Ov i represents the relative speed of the obstacle in the ith sub-time zone, Oa i represents the relative azimuth of the obstacle in the ith sub-time zone, Od i Indicates the obstacle size area of ​​the i-th sub-time zone.

[0015] Preferably, the sensor data analysis method is used to establish a sensor data analysis model, import the sensor data transmitted in step 2 into the sensor data analysis model, and calculate the sensor evaluation value of each sub-time zone, which is specifically expressed as: , D i represents the sensor evaluation value of the i-th sub-time zone, Dr i represents the millimeter-wave radar detection intensity in the ith sub-time zone, Du i represents the ultrasonic sensor echo time of the ith sub-time zone, Dc i represents the camera image clarity of the ith sub-time zone, Dt i Indicates the temperature difference of the infrared sensor in the i-th sub-time zone.

[0016] Preferably, the comprehensive analysis model is specifically expressed as: , η represents the comprehensive abnormality index of the target vehicle, E i represents the environmental assessment value of the ith sub-time zone, V i represents the vehicle state evaluation value of the i-th sub-time zone, O i represents the obstacle evaluation value of the ith sub-time zone, D i represents the sensor evaluation value of the ith sub-time zone, and µ represents other influencing factors of the comprehensive anomaly index.

[0017] Preferably, the preset value of the comprehensive abnormality index is marked as η Def , when η Def ≥η, it means that there is no abnormality in the target vehicle, so the collection and analysis of the target vehicle data is maintained. Def When <η, it indicates that the target vehicle is abnormal, and a warning signal is issued.

[0018] Technical effects and advantages of the present invention: The present invention divides the safety protection time of the target vehicle into multiple sub-time areas through protection time division, which is helpful to accurately monitor and analyze the safety status of the vehicle in different time periods. Through equal time division, it is ensured that each sub-time area can receive balanced attention and processing, thereby improving the continuity and accuracy of safety protection; through data collection, environmental data, vehicle status data, obstacle data and sensor data are comprehensively collected to provide a rich and accurate information basis for subsequent data analysis. The collection of environmental data helps to understand the current road conditions and provide a safety reference for vehicle driving; the collection of vehicle status data can monitor vehicle performance in real time and prevent potential failures; the collection of obstacle data helps to timely discover and avoid potential dangers; the collection of sensor data can verify and supplement other data sources, thereby improving the reliability of overall safety protection; through data analysis, environmental data analysis models, vehicle status data analysis models, obstacle data analysis models and sensor data analysis models are established, and the collected data are deeply analyzed to extract information for safety protection The accuracy and reliability of data analysis results have been significantly improved, providing strong data support for subsequent comprehensive analysis and safety protection; through comprehensive analysis, the environmental assessment value, vehicle status assessment value, obstacle assessment value and sensor assessment value are comprehensively analyzed to calculate the comprehensive abnormality index of the target vehicle, which can comprehensively reflect the current safety status of the vehicle. The comprehensive analysis model takes into account multiple factors, improves the accuracy and comprehensiveness of safety protection, and helps to timely discover and respond to potential safety risks; through safety protection, a preset value of the comprehensive abnormality index is established to judge the comprehensive abnormality index of the target vehicle, which can accurately identify whether the vehicle is abnormal. When the comprehensive abnormality index exceeds the preset value, a warning signal is issued in time to remind the driver or the automatic control system to take necessary protective measures to avoid the occurrence of safety accidents. This side and rear visual safety protection method based on multimodal fusion not only improves the safety of vehicle driving, but also reduces the economic losses and casualties caused by safety accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the overall structure of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] refer to Figure 1 The side and rear visual safety protection method based on multimodal fusion shown includes Step 1: Protection time division: It is used to determine the safety protection time of the target vehicle as the target time area, and divide the target time area into various sub-time areas by equal time division, and mark them as 1, 2...n in sequence.

[0022] Step 2: Data collection: used to collect environmental data, vehicle status data, obstacle data and sensor data in each sub-time zone, and transmit the collected data to step 3.

[0023] The environmental data include current road visibility, ambient light intensity, road slip coefficient and ambient temperature, which are marked as Ev, El, Er and Ea respectively; the vehicle status data include vehicle real-time speed, vehicle steering angle, brake pedal opening and accelerator pedal opening, which are marked as Vs, Va, Vb and Vt respectively; the obstacle data include obstacle relative distance, obstacle relative speed, obstacle relative azimuth and obstacle size area, which are marked as Or, Ov, Oa and Od respectively; the sensor data include millimeter wave radar detection intensity, ultrasonic sensor echo time, camera image clarity and infrared sensor temperature difference, which are marked as Dr, Du, Dc and Dt respectively.

[0024] The current road visibility is measured by emitting a laser beam through a laser visibility sensor to measure the scattering and attenuation of the laser by the atmosphere. The intensity of the scattered light is inversely proportional to the visibility. The ambient light intensity uses a photodiode to convert light energy into an electrical signal, which is measured by a light intensity sensor to automatically adjust the vehicle lighting system and the exposure parameters of the auxiliary driving camera. The road slippery coefficient uses a special friction coefficient sensor and is calculated by measuring the mechanical friction between the tire and the road surface. The ambient temperature is measured using the resistance characteristics of the NTC thermistor that changes with temperature, which affects the engine performance control and air conditioning system adjustment.

[0025] The real-time speed of the vehicle is calculated by detecting the pulse signal generated by the rotation of the gear through the Hall sensor installed on the wheel. The vehicle steering angle uses an angle encoder to detect the steering wheel rotation angle, which can be divided into incremental and absolute types. The brake pedal opening is measured by a linear potentiometer to measure the brake pedal displacement and converted into a percentage value. The accelerator pedal opening uses a non-contact potentiometer to measure the accelerator pedal position for engine power output control.

[0026] The relative distance of the obstacle means that the millimeter-wave radar emits electromagnetic waves, and the distance is calculated by measuring the reflected wave time. The relative speed of the obstacle is based on the Doppler effect, and the relative speed is calculated by measuring the change in the reflected wave frequency. The relative azimuth of the obstacle uses a multi-channel radar array, and the target azimuth is determined by the phase difference of the signals received by adjacent antennas. The size and area of ​​the obstacle are acquired by collecting images with a camera, and the target contour is extracted and the area is calculated using a computer vision algorithm.

[0027] The millimeter wave radar detection strength is measured by measuring the power strength of the radar receiving signal, the ultrasonic sensor echo time is measured by recording the time interval from ultrasonic emission to reception of the reflected wave, the camera image clarity is evaluated by calculating the ratio of signal to noise in the image, and the infrared sensor temperature difference is measured by the infrared sensor. The temperature difference between the target and the environment is used for nighttime pedestrian detection and special object recognition.

[0028] Step 3: Data analysis: including environmental data analysis method, vehicle status data analysis method, obstacle data analysis method and sensor data analysis method, which are used to analyze the data transmitted in step 2 and transmit the analysis results to step 4.

[0029] The environmental data analysis method is used to establish an environmental data analysis model, import the environmental data transmitted in step 2 into the environmental data analysis model, and calculate the environmental assessment value of each sub-time zone, which is specifically expressed as: , E i represents the environmental assessment value of the ith sub-time zone, Ev i represents the current road visibility in the ith sub-time zone, El i represents the ambient light intensity of the ith sub-time zone, Er i Indicates the road slip coefficient of the ith sub-time zone, Ea i Represents the ambient temperature of the i-th sub-time zone.

[0030] The vehicle state data analysis method is used to establish a vehicle state data analysis model, import the vehicle state data transmitted in step 2 into the vehicle state data analysis model, and calculate the vehicle state evaluation value of each sub-time zone, which is specifically expressed as: , V i represents the vehicle state evaluation value of the i-th sub-time zone, Vs i represents the real-time speed of the vehicle in the ith sub-time zone, Va i represents the vehicle steering angle in the ith sub-time zone, Vb i represents the brake pedal opening in the ith sub-time zone, Vt i Indicates the accelerator pedal opening degree in the i-th sub-time zone.

[0031] The obstacle data analysis method is used to establish an obstacle data analysis model, import the obstacle data transmitted in step 2 into the obstacle data analysis model, and calculate the obstacle evaluation value of each sub-time zone, which is specifically expressed as: , O i represents the obstacle assessment value of the ith sub-time zone, Or i represents the relative distance of obstacles in the ith sub-time zone, Ov i represents the relative speed of the obstacle in the ith sub-time zone, Oa i represents the relative azimuth of the obstacle in the ith sub-time zone, Od i Indicates the obstacle size area of ​​the i-th sub-time zone.

[0032] The sensor data analysis method is used to establish a sensor data analysis model, import the sensor data transmitted in step 2 into the sensor data analysis model, and calculate the sensor evaluation value of each sub-time zone, which is specifically expressed as: , D i represents the sensor evaluation value of the i-th sub-time zone, Dr i represents the millimeter-wave radar detection intensity in the ith sub-time zone, Du i represents the ultrasonic sensor echo time of the ith sub-time zone, Dc i represents the camera image clarity of the ith sub-time zone, Dt i Indicates the temperature difference of the infrared sensor in the i-th sub-time zone.

[0033] Step 4: Comprehensive analysis: The data transmitted in step 3 is comprehensively analyzed through comprehensive analysis, and the analysis results are transmitted to step 5.

[0034] The comprehensive analysis model is specifically expressed as: , η represents the comprehensive abnormality index of the target vehicle, E i represents the environmental assessment value of the ith sub-time zone, V i represents the vehicle state evaluation value of the i-th sub-time zone, O i represents the obstacle evaluation value of the ith sub-time zone, D i represents the sensor evaluation value of the ith sub-time zone, and µ represents other influencing factors of the comprehensive anomaly index.

[0035] Step 5: Safety protection: The comprehensive abnormality index of the target vehicle is judged by establishing a preset value of the comprehensive abnormality index, and a control instruction is issued according to the judgment result.

[0036] The preset value of the comprehensive abnormality index is marked as η Def , when η Def≥η, it means that there is no abnormality in the target vehicle, so the collection and analysis of the target vehicle data is maintained. Def When <η, it indicates that the target vehicle is abnormal, and a warning signal is issued.

[0037] The present invention divides the safety protection time of the target vehicle into multiple sub-time areas through protection time division, which is helpful to accurately monitor and analyze the safety status of the vehicle in different time periods. Through equal time division, it is ensured that each sub-time area can receive balanced attention and processing, thereby improving the continuity and accuracy of safety protection; through data collection, environmental data, vehicle status data, obstacle data and sensor data are comprehensively collected to provide a rich and accurate information basis for subsequent data analysis. The collection of environmental data helps to understand the current road conditions and provide a safety reference for vehicle driving; the collection of vehicle status data can monitor vehicle performance in real time and prevent potential failures; the collection of obstacle data helps to timely discover and avoid potential dangers; the collection of sensor data can verify and supplement other data sources, thereby improving the reliability of overall safety protection; through data analysis, environmental data analysis models, vehicle status data analysis models, obstacle data analysis models and sensor data analysis models are established, and the collected data are deeply analyzed to extract information for safety protection The accuracy and reliability of data analysis results have been significantly improved, providing strong data support for subsequent comprehensive analysis and safety protection; through comprehensive analysis, the environmental assessment value, vehicle status assessment value, obstacle assessment value and sensor assessment value are comprehensively analyzed to calculate the comprehensive abnormality index of the target vehicle, which can comprehensively reflect the current safety status of the vehicle. The comprehensive analysis model takes into account multiple factors, improves the accuracy and comprehensiveness of safety protection, and helps to timely discover and respond to potential safety risks; through safety protection, a preset value of the comprehensive abnormality index is established to judge the comprehensive abnormality index of the target vehicle, which can accurately identify whether the vehicle is abnormal. When the comprehensive abnormality index exceeds the preset value, a warning signal is issued in time to remind the driver or the automatic control system to take necessary protective measures to avoid the occurrence of safety accidents. This side and rear visual safety protection method based on multimodal fusion not only improves the safety of vehicle driving, but also reduces the economic losses and casualties caused by safety accidents.

[0038] Secondly: In the drawings of the embodiments disclosed in the present invention, only the structures related to the embodiments disclosed in the present invention are involved, and other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of the present invention can be combined with each other; Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A side and rear visual safety protection method based on multimodal fusion, characterized in that: include: Step 1: Protection time division: used to determine the safety protection time of the target vehicle as the target time area, divide the target time area into various sub-time areas by equal time division, and mark them as 1, 2...n in sequence; Step 2: Data collection: used to collect environmental data, vehicle status data, obstacle data and sensor data in each sub-time zone, and transmit the collected data to step 3; Step 3: Data analysis: including an environmental data analysis method, a vehicle status data analysis method, an obstacle data analysis method, and a sensor data analysis method, for analyzing the data transmitted in step 2, and transmitting the analysis results to step 4; Step 4: Comprehensive analysis: The data transmitted in step 3 is comprehensively analyzed through comprehensive analysis, and the analysis results are transmitted to step 5; Step 5: Safety protection: The comprehensive abnormality index of the target vehicle is judged by establishing a preset value of the comprehensive abnormality index, and a control instruction is issued according to the judgment result.

2. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized in that: The environmental data include current road visibility, ambient light intensity, road slip coefficient and ambient temperature, which are marked as Ev, El, Er and Ea respectively; the vehicle status data include vehicle real-time speed, vehicle steering angle, brake pedal opening and accelerator pedal opening, which are marked as Vs, Va, Vb and Vt respectively; the obstacle data include obstacle relative distance, obstacle relative speed, obstacle relative azimuth and obstacle size area, which are marked as Or, Ov, Oa and Od respectively; the sensor data include millimeter wave radar detection intensity, ultrasonic sensor echo time, camera image clarity and infrared sensor temperature difference, which are marked as Dr, Du, Dc and Dt respectively.

3. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized in that: The environmental data analysis method is used to establish an environmental data analysis model, import the environmental data transmitted in step 2 into the environmental data analysis model, and calculate the environmental assessment value of each sub-time zone, which is specifically expressed as: , E i represents the environmental assessment value of the ith sub-time zone, Ev i represents the current road visibility in the ith sub-time zone, El i represents the ambient light intensity of the ith sub-time zone, Er i Indicates the road slip coefficient of the ith sub-time zone, Ea i Represents the ambient temperature of the i-th sub-time zone.

4. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized in that: The vehicle state data analysis method is used to establish a vehicle state data analysis model, import the vehicle state data transmitted in step 2 into the vehicle state data analysis model, and calculate the vehicle state evaluation value of each sub-time zone, which is specifically expressed as: , V i represents the vehicle state evaluation value of the i-th sub-time zone, Vs i represents the real-time speed of the vehicle in the ith sub-time zone, Va i represents the vehicle steering angle in the ith sub-time zone, Vb i represents the brake pedal opening in the ith sub-time zone, Vt i Indicates the accelerator pedal opening degree in the i-th sub-time zone.

5. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized in that: The obstacle data analysis method is used to establish an obstacle data analysis model, import the obstacle data transmitted in step 2 into the obstacle data analysis model, and calculate the obstacle evaluation value of each sub-time zone, which is specifically expressed as: , O i represents the obstacle assessment value of the ith sub-time zone, Or i represents the relative distance of obstacles in the ith sub-time zone, Ov i represents the relative speed of the obstacle in the ith sub-time zone, Oa i represents the relative azimuth of the obstacle in the ith sub-time zone, Od i Indicates the obstacle size area of ​​the i-th sub-time zone.

6. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized in that: The sensor data analysis method is used to establish a sensor data analysis model, import the sensor data transmitted in step 2 into the sensor data analysis model, and calculate the sensor evaluation value of each sub-time zone, which is specifically expressed as: , D i represents the sensor evaluation value of the i-th sub-time zone, Dr i represents the millimeter-wave radar detection intensity in the ith sub-time zone, Du i represents the ultrasonic sensor echo time of the ith sub-time zone, Dc i represents the camera image clarity of the ith sub-time zone, Dt i Indicates the temperature difference of the infrared sensor in the i-th sub-time zone.

7. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized by: The comprehensive analysis model is specifically expressed as: , η represents the comprehensive abnormality index of the target vehicle, E i represents the environmental assessment value of the ith sub-time zone, V i represents the vehicle state evaluation value of the i-th sub-time zone, O i represents the obstacle evaluation value of the ith sub-time zone, D i represents the sensor evaluation value of the ith sub-time zone, and µ represents other influencing factors of the comprehensive anomaly index.

8. The side and rear visual safety protection method based on multimodal fusion according to claim 1 is characterized by: The preset value of the comprehensive abnormality index is marked as η Def , when η Def ≥η, it means that there is no abnormality in the target vehicle, so the collection and analysis of the target vehicle data is maintained. Def When <η, it indicates that the target vehicle is abnormal, and a warning signal is issued.

Images