Intelligent driving vehicle emergency response method for emergencies inside and outside the vehicle

Abstract

The present invention discloses an intelligent driving vehicle emergency response method for emergencies inside and outside the vehicle, which belongs to the field of intelligent driving technology. The method includes: obtaining the polarization intensity of the outer glass curtain wall of the intelligent vehicle at different reflection angles in real time, and generating a polarization state distribution map based on the polarization intensity. The polarization intensity of different reflection intensities and incident angles is calibrated based on the polarization state distribution map by an adaptive polarization filter component to reduce the mirror glare component. Multiple sensors are used to synchronously collect multimodal sensing data in the mirror reflection failure area, and the distorted area in the optical image is repaired and compensated to generate a dynamic mask in the mirror reflection area. Based on the multimodal sensing data, the dynamic mask is fused to obtain a fusion feature map, and an event list is identified according to a preset event template. In response to the event list, the emergency response strategy is retrieved according to the priority sequence, and the corresponding intelligent driving instructions are executed according to the emergency response strategy.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to an intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle. Background Art

[0002] Intelligent driving vehicles add advanced sensors (radar, camera), controllers, actuators and other devices to ordinary vehicles. Through on-board sensing systems and information terminals, they realize intelligent information exchange with people, vehicles, roads, etc., so that vehicles have intelligent environmental perception capabilities, can automatically analyze the safety and danger status of vehicle driving, and make the vehicle reach the destination according to the wishes of the people, ultimately achieving the purpose of replacing human operation to reduce the burden of human driving.

[0003] In existing technologies, the overall control system of an intelligent driving vehicle collects data from various subsystems and processes this data in a unified manner to control the intelligent driving vehicle. For example, acquired video images of the road environment can be statistically analyzed to establish a database for identifying urban road scenes, rural road scenes, and highway scenes. A deep convolutional neural network is then used to extract features and perform convolution training on sample images in the database to generate a convolutional neural network classifier. Ultimately, the real-time perception images are input into the convolutional neural network classifier for recognition, thereby classifying the current driving scene in which the vehicle is located. However, when an intelligent driving vehicle is driving in a central business district surrounded by high-rise buildings, the glass curtain walls on both sides strongly reflect light, causing dramatic changes in pitch angle. In urban canyon glass curtain wall scenes, due to various lighting factors such as glass refraction, it is very easy for the intelligent driving vehicle's camera, radar, and other sensor equipment to struggle to obtain accurate sensor data, resulting in numerous data monitoring blind spots. This makes it difficult for the intelligent driving system to accurately identify the current driving scene, affecting the intelligent driving process. Summary of the Invention

[0004] In order to address the deficiencies in the prior art, the purpose of the present invention is to solve the above-mentioned defects and further propose an intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle.

[0005] The present invention adopts the following technical solutions.

[0006] A first aspect of the present invention discloses an intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle, the method comprising:

[0007] Acquire the polarization intensity of the glass curtain wall outside the intelligent vehicle at different reflection angles in real time, and generate a polarization state distribution diagram based on the polarization intensity;

[0008] Calibrate the polarization intensity of different reflection intensities and incident angles based on the polarization state distribution diagram through an adaptive polarization filter component to reduce the specular glare component;

[0009] Multiple sensors are used to synchronously collect multimodal sensing data in the mirror reflection failure area, and the distorted areas in the optical image are repaired and compensated to generate a dynamic mask in the mirror reflection area.

[0010] Based on the multimodal sensing data, the dynamic mask is fused to obtain a fused feature map, and an event list is identified according to a preset event template;

[0011] Retrieving an emergency response strategy in a priority sequence in response to the event list, and executing a corresponding intelligent driving instruction according to the emergency response strategy;

[0012] The multimodal sensing data is divided into optical images, radar point clouds and ultrasonic echoes, and the event list includes event types and event three-dimensional positions.

[0013] Furthermore, the real-time acquisition of polarization intensity of the outer glass curtain wall of the intelligent vehicle at different reflection angles and the generation of a polarization state distribution diagram based on the polarization intensity include:

[0014] Obtaining a sampling timestamp sequence of the camera, and controlling the polarization sensor to resample by linear interpolation according to the sampling timestamp sequence to obtain a unified timing sequence and multi-angle light intensity after time alignment;

[0015] Collecting a pixel-level light intensity array aligned with the camera through a plurality of polarization filters and an imaging array on the polarization sensor, and constructing a polarization intensity matrix based on the pixel-level light intensity array;

[0016] Calculating the polarization angle corresponding to each pixel position in the polarization intensity matrix by Stokes component difference to construct a polarization angle matrix;

[0017] Pseudo-color mapping is performed on the polarization angle matrix, and the brightness is adjusted based on the sum of the multi-angle illumination intensities of each pixel position to generate the polarization state distribution map.

[0018] Furthermore, the adaptive polarization filter assembly calibrates the polarization intensity of different reflection intensities and incident angles based on the polarization state distribution diagram to reduce the specular glare component, including:

[0019] Performing a weighted average of the polarization angle matrix and the sum of the multi-angle light intensity at each pixel position to determine the global optimal filtering angle, and selecting a wave plate according to the current dispersion degree and reflection spectrum width to obtain a filtering phase delay value;

[0020] Determining filter parameters based on the global optimal filtering angle and the filtering phase delay value, and sending a rotation instruction to the stepper motor through the single-chip microcomputer to control the filter to rotate to the global optimal filtering angle;

[0021] Output voltage to the liquid crystal delay unit through the DAC to achieve the filtering phase delay value, and return the corresponding actual filtering angle and actual filtering phase delay value in real time through the encoder and voltage sensor;

[0022] The adjusted polarization state distribution diagram, light intensity matrix and reflection mask are obtained according to the actual filtering angle and the actual filtering phase delay value, the light intensity ratio of the mask area to the full field of view is calculated, and the residual glare ratio is output.

[0023] Furthermore, the method of synchronously collecting multimodal sensing data in the mirror reflection failure area using multiple sensors and repairing and compensating the distorted area in the optical image to generate a dynamic mask in the mirror reflection area includes:

[0024] The microcontroller monitors the start time of the calibrated unified timing sequence, sends a standardized trigger frame containing a timestamp field on the CAN bus, and outputs a bus trigger instruction;

[0025] The bus trigger instruction is sent to the camera. The camera performs sampling with fixed exposure parameters in response to the bus trigger instruction and reads the pixel array using a global shutter mode to obtain an optical image frame.

[0026] Furthermore, the method of synchronously collecting multimodal sensing data in the mirror reflection failure area using multiple sensors and repairing and compensating the distorted area in the optical image to generate a dynamic mask in the mirror reflection area also includes:

[0027] The bus trigger instruction is sent to the millimeter wave radar unit. The millimeter wave radar unit transmits an FMCW detection signal at a fixed pulse frame rate in response to the bus trigger instruction, and samples the echo on the receiving array to generate a point cloud matrix.

[0028] Sending the bus trigger instruction to the ultrasound unit, the ultrasound unit sending pulses along a fixed frequency in response to the bus trigger instruction and collecting echo samples on the earpiece array to generate an echo sequence;

[0029] The central fusion unit receives the point cloud matrix and echo sequence from the millimeter wave radar unit and the ultrasonic unit, and fuses and encapsulates the point cloud matrix, echo sequence and optical image based on the trigger timestamp to obtain a multimodal data packet.

[0030] Furthermore, the method of synchronously collecting multimodal sensing data in the mirror reflection failure area using multiple sensors and repairing and compensating the distorted area in the optical image to generate a dynamic mask in the mirror reflection area also includes:

[0031] The camera and radar are calibrated with external parameters using a calibration plate to obtain the rotation matrix and translation vector between the camera and radar, and the radar point cloud is mapped to the pixel coordinate system.

[0032] Obtain the minimum depth value of all valid mapping points, combine bilinear interpolation to generate a depth map, and project the ultrasonic echo intensity to the corresponding pixel position according to the rigid connection parameters between the ultrasonic sensor and the camera to obtain an echo intensity map;

[0033] Based on the depth map and the echo intensity map, the reflectance index corresponding to each pixel coordinate is calculated in combination with the Stokes component difference matrix, and the adaptive threshold is calculated according to the full-image statistical characteristics of the reflectance index;

[0034] Each pixel is subjected to binary processing to obtain a binary mask, and the binary mask is smoothed by using a median filter to generate the dynamic mask and the corresponding pixel weight map.

[0035] Furthermore, the dynamic mask is fused based on the multimodal sensing data to obtain a fused feature map, and an event list is identified according to a preset event template, including:

[0036] Performing weighted preprocessing on each pixel in the multimodal sensing data located in a strong reflection area according to weight attenuation, combining a dynamic mask and a pixel weight map, to obtain weighted multimodal data;

[0037] Performing weighted fusion on the weighted multimodal data according to a preset ratio to obtain a fused feature map, and performing local feature extraction on the fused feature map through a convolutional feature extraction network to obtain a local feature map;

[0038] Calculate the window cosine similarity based on each preset event template in the preset event template library, and retain the event templates that meet the candidate conditions as candidate templates to obtain a candidate event list;

[0039] Each template index in the candidate event list is mapped to a specific event type, and the final event list is identified by combining the timestamp and the three-dimensional position of the event.

[0040] Furthermore, the retrieving an emergency response strategy according to a priority sequence in response to the event list, and executing a corresponding intelligent driving instruction according to the emergency response strategy, includes:

[0041] Calculating a priority score for each event in the event list, and sorting the events in the event list according to the priority scores, so as to retrieve a policy template corresponding to each event in the sorted event list;

[0042] Calculating the vehicle speed, driving angle, and instruction execution time according to the strategy template, and generating the intelligent driving instruction based on the vehicle speed, driving angle, and instruction execution time;

[0043] The intelligent driving instruction is sent to the intelligent driving vehicle terminal through the CAN bus, and the vehicle driving status is obtained in real time when the intelligent driving vehicle executes the intelligent driving instruction.

[0044] A second aspect of the present invention provides an intelligent vehicle emergency response device for sudden situations inside and outside the vehicle, which is used to implement the intelligent vehicle emergency response method for sudden situations inside and outside the vehicle as described in any one of the first aspects, and the device includes:

[0045] A polarization monitoring module is used to obtain the polarization intensity of the outer glass curtain wall of the intelligent vehicle at different reflection angles in real time, and generate a polarization state distribution diagram based on the polarization intensity;

[0046] a specular glare reduction module, configured to calibrate the polarization intensity at different reflection intensities and incident angles based on the polarization state distribution diagram through an adaptive polarization filter assembly, so as to reduce the specular glare component;

[0047] A data calibration module is used to synchronously collect multimodal sensing data in the mirror reflection failure area using multiple sensors, and to repair and compensate the distorted areas in the optical image to generate a dynamic mask in the mirror reflection area;

[0048] A feature fusion and event recognition module, configured to fuse the dynamic mask based on the multimodal sensing data to obtain a fused feature map, and identify an event list according to a preset event template;

[0049] An instruction issuing module, configured to retrieve an emergency response strategy in accordance with a priority sequence in response to the event list, and execute a corresponding intelligent driving instruction according to the emergency response strategy;

[0050] The multimodal sensing data is divided into optical images, radar point clouds and ultrasonic echoes, and the event list includes event types and event three-dimensional positions.

[0051] A third aspect of the present invention discloses a terminal, comprising a processor and a storage medium;

[0052] The storage medium is used to store instructions;

[0053] The processor is configured to operate according to the instructions to execute the steps of the method of the first aspect.

[0054] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0055] The beneficial effects of the present invention are that, compared with the prior art, the present invention has the following advantages:

[0056] (1) The present invention captures the polarization disturbance of the glass curtain wall at different reflection angles in real time, providing a precise quantitative basis for subsequent filtering adjustment, which can effectively avoid the visual information fault caused by optical mirror reflection. Secondly, by utilizing the penetration and echo advantages of radar and ultrasound in the mirror reflection failure area, the blind spot of a single visual sensor is supplemented, and multi-angle and cross-physical domain information acquisition of the same scene is achieved. This allows intelligent driving vehicles to obtain more accurate intelligent driving scene data to the greatest extent possible even in scenes where the glass curtain walls on both sides strongly reflect light and the pitch angle changes drastically, reducing the data monitoring blind spot caused by light reflection and ensuring the smooth progress of the intelligent driving process.

[0057] (2) The present invention reduces specular glare components in real time in scenes with varying reflection intensities and incident angles, restoring usable pure scene optical information and effectively avoiding the technical defect of a "visual vacuum zone" caused by the lag in traditional fixed filter adjustment. Furthermore, by precisely demarcating the distorted areas in the image, the subsequent image restoration and multi-source fusion steps can specifically ignore or compensate for the deformed pixels, eliminating the possibility of strong reflection artifacts misleading the event detection model and further ensuring the smooth progress of intelligent driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention;

[0059] Figure 2 It is a structural schematic diagram of the intelligent driving vehicle emergency response device provided by the present invention for sudden situations inside and outside the vehicle. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0061] like Figure 1 As shown, in one embodiment, a method for intelligent driving vehicle emergency response to sudden situations inside and outside the vehicle includes the following steps:

[0062] Step S110 , obtaining the polarization intensity of the outer glass curtain wall of the smart vehicle at different reflection angles in real time, and generating a polarization state distribution diagram based on the polarization intensity.

[0063] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S110 specifically includes the following steps:

[0064] Step S111 , obtaining a sampling timestamp sequence of the camera, and controlling the polarization sensor to resample by linear interpolation according to the sampling timestamp sequence, so as to obtain a unified timing sequence and multi-angle illumination intensity after time alignment.

[0065] Step S112 , collecting a pixel-level light intensity array aligned with the camera through multiple polarization filters and an imaging array on the polarization sensor, and constructing a polarization intensity matrix based on the pixel-level light intensity array.

[0066] Step S113 , calculating the polarization angle corresponding to each pixel position in the polarization intensity matrix by using Stokes component difference to construct a polarization angle matrix.

[0067] Step S114 , performing pseudo color mapping on the polarization angle matrix, and adjusting the brightness based on the sum of the multi-angle illumination intensities of each pixel position to generate a polarization state distribution map.

[0068] In a specific embodiment, the present invention provides an intelligent vehicle emergency response method for sudden situations inside and outside the vehicle, including steps 1 to 6:

[0069] Step 1: Dynamic measurement of the ambient polarization state.

[0070] Real-time capture of polarization disturbances of glass curtain walls at different reflection angles provides precise quantitative basis for subsequent filtering adjustments, effectively avoiding visual information discontinuities caused by optical mirror reflections.

[0071] The following steps are involved:

[0072] Step 1.1, timing synchronization and correction.

[0073] Specifically, based on the camera timing, linear interpolation is used to resample the polarization sensor data to the same time point to obtain an aligned unified timing sequence to ensure that the optical image and the polarization light intensity sample are at the same time base.

[0074] Among them, for each arbitrary time point in the unified time series ,make:

[0075]

[0076] Where, express The polarization filter angle at this moment is Light intensity readings at 1000 s; is the sampling timestamp sequence of the polarization sensor before resampling. The time points in this sequence are represented as .

[0077] Step 1.2: Multi-angle light intensity collection.

[0078] Specifically, the polarization sensor consists of four polarization filters (angles 0°, 45°, 90°, and 135°) and an imaging array of the same size. It directly returns a pixel-level light intensity array aligned with the camera to obtain the original light intensity at each polarization angle under the same field of view, providing a data basis for feature extraction.

[0079] Step 1.3, pixel-level polarization angle calculation.

[0080] Specifically, for each pixel (i, j), the polarization angle of each pixel is calculated based on the Stokes component difference , the expression is:

[0081]

[0082] Where, Indicates the polarization filter angle is The light intensity of the pixel (i, j) at time (cd / ㎡), The range is 0-180°. The four-way light intensity is mapped to the spatially distributed polarization angle to intuitively distinguish between the specular reflection and diffuse reflection areas. The polarization angle matrix can be constructed based on all pixel polarization angles.

[0083] Step 1.4: Generate high-resolution polarization state distribution map.

[0084] Specifically, the polarization angle matrix obtained in step 1.3 is pseudo-colored, and the brightness is adjusted using a matrix consisting of the sum of the light intensities at all angles of the corresponding pixels (angles 0°, 45°, 90°, and 135°). Finally, an intuitive visual graph is formed to display the polarization characteristic intensity of each region, assist in determining the position of strong specular reflections, and provide a precise target for online filter calibration.

[0085] Step S120 , calibrating the polarization intensity of different reflection intensities and incident angles based on the polarization state distribution diagram by using an adaptive polarization filter component to reduce the specular glare component.

[0086] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S120 specifically includes the following steps:

[0087] Step S121 , performing weighted averaging on the polarization angle matrix and the sum of the multi-angle illumination intensities at each pixel position to determine the global optimal filtering angle, and selecting a wave plate according to the current dispersion degree and reflection spectrum width to obtain a filtering phase delay value.

[0088] Step S122: determine the filter parameters based on the global optimal filtering angle and the filtering phase delay value, and send a rotation instruction to the stepper motor through the single chip microcomputer to control the filter to rotate to the global optimal filtering angle.

[0089] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S120 specifically further includes the following steps:

[0090] In step S123 , the DAC outputs a voltage to the liquid crystal delay unit to achieve a filtering phase delay value, and the encoder and the voltage sensor transmit back the corresponding actual filtering angle and actual filtering phase delay value in real time.

[0091] Step S124 , obtaining the adjusted polarization state distribution diagram, light intensity matrix, and reflection mask according to the actual filtering angle and the actual filtering phase delay value, calculating the light intensity ratio of the mask area to the full field of view, and outputting the residual glare ratio.

[0092] In a specific embodiment, the present invention provides an intelligent vehicle emergency response method for sudden situations both inside and outside the vehicle. Step 2 involves online calibration of the adaptive polarization filter assembly. This method reduces specular glare in real time in scenarios with varying reflection intensities and incident angles, restoring usable pure scene optical information and avoiding the "visual vacuum" caused by the lag in adjustment of traditional fixed filters.

[0093] The following steps are involved:

[0094] Step 2.1: Polarization angle weighted averaging.

[0095] Specifically, the weighted average is calculated based on the polarization angle matrix and the pixel total intensity matrix obtained in step 1. The expression is:

[0096]

[0097] Where, The sum of the light intensities at four angles of pixel (i, j) is within the range of 0.100 cd / ㎡; is the global optimal filtering angle.

[0098] This step is used to obtain the most representative polarization direction of the current full field of view, providing a precise target for mechanical rotation of the filter.

[0099] Step 2.2: Phase delay quantization setting.

[0100] Specifically, a quarter-wave spectrum is selected according to the dispersion degree and the reflection spectrum width in the scene, which is expressed as:

[0101]

[0102] Where, The filter phase delay value can be determined by looking up the table within [0.4π, 0.6π] according to the on-site spectral band range (400-700) if fine-tuning is required.

[0103] Step 2.3: Perform drive and feedback acquisition.

[0104] Specifically, first determine the filter parameters, that is, the filtering angle and filter phase delay value , the microcontroller sends a rotation command to the stepper motor to rotate the filter to the filtering angle , synchronously output voltage to the liquid crystal delay unit through the DAC to achieve the filtering phase delay value The encoder and voltage sensor transmit the actual filter angle and filter phase delay values ​​in real time. This step converts the calculated parameters into physical adjustments and ensures calibration accuracy through closed-loop feedback.

[0105] Step 2.4: Calibration effect evaluation and iteration.

[0106] Specifically, based on step 2.3, adjust the parameters to re-obtain the corresponding polarization state distribution diagram and light intensity matrix As well as the reflective mask M, the light intensity ratio of the mask area to the full field of view is calculated using the expression:

[0107]

[0108] Where, is the residual glare ratio, if If the value is greater than 0.1, return to step 2.1 and recalibrate. Otherwise, complete the calibration. This step quantifies the effect of specular reflection suppression and ensures that no new blind spots are left after light suppression.

[0109] In step S130 , a plurality of sensors are used to synchronously collect multimodal sensing data in the mirror reflection failure area, and the distorted area in the optical image is repaired and compensated to generate a dynamic mask in the mirror reflection area.

[0110] Among them, multimodal sensing data is divided into optical images, radar point clouds and ultrasonic echoes.

[0111] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S130 specifically includes the following steps:

[0112] Step S131 , monitoring the start time of the calibrated unified timing sequence through the single chip microcomputer, sending a standardized trigger frame including a timestamp field on the CAN bus, and outputting a bus trigger instruction.

[0113] In step S132 , the bus trigger instruction is sent to the camera. The camera responds to the bus trigger instruction by sampling with fixed exposure parameters and reading the pixel array using a global shutter mode to obtain an optical image frame.

[0114] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S130 specifically further includes the following steps:

[0115] In step S133, the bus trigger instruction is sent to the millimeter wave radar unit. In response to the bus trigger instruction, the millimeter wave radar unit transmits an FMCW detection signal at a fixed pulse frame rate and samples the echo on the receiving array to generate a point cloud matrix.

[0116] Step S134: sending a bus trigger instruction to the ultrasonic unit. The ultrasonic unit sends pulses along a fixed frequency in response to the bus trigger instruction and collects echo samples on the earpiece array to generate an echo sequence.

[0117] In step S135 , the central fusion unit receives the point cloud matrix and echo sequence from the millimeter wave radar unit and the ultrasonic unit, and fuses and encapsulates the point cloud matrix, echo sequence, and optical image based on the trigger timestamp to obtain a multimodal data packet.

[0118] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S130 specifically further includes the following steps:

[0119] In step S136, the camera and radar are calibrated with external parameters using a calibration plate to obtain the rotation matrix and translation vector between the camera and radar, and the radar point cloud is mapped to a pixel coordinate system.

[0120] Step S137: Obtain the minimum depth value of all valid mapping points, generate a depth map by combining bilinear interpolation, and project the ultrasonic echo intensity to the corresponding pixel position according to the rigid connection parameters between the ultrasonic sensor and the camera to obtain an echo intensity map.

[0121] Step S138 : Based on the depth map and the echo intensity map, the reflection index corresponding to each pixel coordinate is calculated in combination with the Stokes component difference matrix, and the adaptive threshold is calculated according to the full-image statistical characteristics of the reflection index.

[0122] In step S139 , each pixel is subjected to binary processing to obtain a binary mask, and the binary mask is smoothed using a median filter to generate a dynamic mask and a corresponding pixel weight map.

[0123] In a specific embodiment, the present invention provides an intelligent vehicle emergency response method for sudden situations both inside and outside the vehicle. Step 3 involves synchronously collecting multimodal sensor data. This method leverages the penetration and echo advantages of radar and ultrasound in areas where mirror reflection fails, complementing the blind spots of a single visual sensor and enabling multi-angle, cross-physical domain information acquisition of the same scene.

[0124] The following steps are involved:

[0125] Step 3.1, synchronize the trigger signal.

[0126] Specifically, after the microcontroller detects the completion of step 2 calibration, it begins reading the start time of the unified timing sequence and sends a standardized trigger frame containing a timestamp field on the system bus (CAN or Ethernet). Upon receiving the frame, each sensor (camera, millimeter-wave radar, and ultrasonic unit) immediately enters data acquisition mode to ensure a synchronized response at the next instant. This step provides a single, unified hardware trigger path, eliminating latency differences caused by separate triggering of each sensor and ensuring synchronized sampling across domains.

[0127] Step 3.2, optical image capture.

[0128] Specifically, after receiving a trigger command (including a timestamp), the forward-facing camera begins sampling with fixed exposure parameters (typically 1-5ms), using global shutter mode. Immediately after a single exposure, the pixel array is read to generate an optical image frame, along with metadata fields. The generated optical image frame is then sent to the processing unit via a high-speed bus (GigE Vision or CSI). This step enables the acquisition of visual information free of polarization glare at the precise moment, providing clear images for target detection and depth estimation.

[0129] Step 3.3: Millimeter wave radar and ultrasonic wave synchronous sampling.

[0130] Specifically, after receiving a trigger command, the millimeter-wave radar transmits an FMCW detection signal at a fixed pulse frame rate (e.g., 50 Hz) and simultaneously samples the echoes on the receiving array, generating a point cloud matrix consisting of multiple reflection points with three-dimensional coordinates and echo intensities. After receiving the trigger command, the ultrasonic unit sends a pulse at a fixed frequency (e.g., 40 Hz) and samples the echoes on the earpiece array, generating an echo sequence consisting of multiple sampling point amplitudes. The radar and ultrasonic units then send the data corresponding to the trigger timestamp to a central fusion unit. This step leverages the penetrating and high-precision ranging capabilities of millimeter-wave radar, combined with ultrasonic waves' sensitivity to low-reflectivity materials, to further supplement optical blind spot data in highly reflective areas.

[0131] Step 3.4: Data packet and timing mark.

[0132] Specifically, the central fusion unit aligns the three data streams—the optical image, radar point cloud, and ultrasonic echo—with a common timestamp and encapsulates them into the required structure. The data transmission delay is calculated and recorded for subsequent delay compensation during fusion. Finally, the encapsulated multimodal data packet is stored in a circular queue, triggering the fusion algorithm for further processing. This step forms a timestamp-indexed snapshot of synchronized multimodal data, providing a unified data unit for accurate fusion and subsequent event monitoring.

[0133] Step 4: Generate dynamic mask of mirror reflection area.

[0134] Accurately divide the distorted areas in the image so that subsequent image restoration and multi-source fusion steps can specifically ignore or compensate for deformed pixels, eliminating strong reflection artifacts that mislead the event detection model.

[0135] The following steps are involved:

[0136] Step 4.1: Multimodal data spatial registration.

[0137] Specifically, first, through calibration of the calibration plate, the rotation matrix R and translation vector t between the camera and the radar are obtained, so that the radar point cloud is mapped to the pixel coordinate system, and the external parameter calibration of the camera and the radar is realized, which is expressed as:

[0138]

[0139] Where, is the pixel coordinate after mapping, is the pixel coordinate before mapping, K is the camera intrinsic parameter matrix, and T is the transpose of the matrix.

[0140] Afterwards, for all valid mapping points The minimum depth value is implanted, and bilinear interpolation is used for the remaining pixels to generate a depth map. Based on the rigid connection parameters between the ultrasonic sensor and the camera, the ultrasonic echo intensity is projected onto the corresponding pixel position to achieve ultrasonic echo mapping. This step establishes the spatial correspondence between optical pixels and radar and ultrasonic data, laying the foundation for subsequent pixel-level multi-source consistency monitoring.

[0141] Step 4.2, calculate the polarization and intensity combined reflection index.

[0142] Specifically, according to the Stokes component difference matrix calculated above, for each pixel (i, j) the polarization intensity and depth are combined , normalized ultrasound echo intensity , defining the reflection index , expressed as:

[0143]

[0144] Where, is the Stokes component difference matrix The value of for The value of , α is the depth attenuation coefficient, and β is the echo compensation coefficient.

[0145] Among them, the reflection index The first term reflects the intensity of polarized glare, the second term attenuates with depth to prevent misidentification of distant reflections, and the third term utilizes the ultrasonic echo reflection area to suppress misidentification. This step generates a grayscale image that comprehensively reflects the likelihood of specular reflection, integrating all physical domain information into a quantified reflectance metric that can be thresholded.

[0146] Step 4.3: Binarization of the reflection area and generation of preliminary mask.

[0147] Specifically, based on the full-image statistical characteristics corresponding to the reflectance index obtained in step 4.2, the Otsu threshold S is calculated to minimize the intra-class variance. For each pixel, if its reflectance index is greater than or equal to threshold S, the initial binary mask is "1", otherwise it is "0", achieving binarization. Finally, a 3×3 median filter is used to perform preliminary smoothing on the binary mask to remove isolated noise points. This step converts the grayscale reflectance index map into preliminary specular reflection area markers, providing the basis for the binary mask.

[0148] Step 4.4, mask morphological refinement and weight map generation.

[0149] Specifically, a closing operation (dilation followed by erosion) with a 5×5 structural element size is applied to the preliminary mask obtained in step 4.3 to fill small holes and connect adjacent regions, resulting in the morphological closing result of the preliminary mask. Next, a 3×3 opening operation (erosion followed by dilation) is applied to the morphological closing result of the preliminary mask to remove narrow necks or short artifacts, resulting in the corresponding dynamic mask. Finally, a weight map is calculated for all pixels with a dynamic mask value of "1", using the normalized reflectance index as the weight. This step ensures that the mask area is coherent and free of isolated points through morphological refinement, generating a pixel-level weight map for subsequent weighted processing based on intensity differences during fusion.

[0150] Step 4.5, dynamic mask output and update trigger.

[0151] Specifically, the dynamic mask and weight map obtained in step 4.4 are packaged with timestamps to form a unified data structure. This data is then pushed to the multi-source fusion and event recognition unit via a high-speed bus to trigger the next fusion calculation. Simultaneously, the system monitors environmental changes (such as lighting, speed, and tunnel switching). When calibration parameters or reflectivity index change significantly, a "MaskStale" signal is reported, triggering the re-execution of steps 4.1 through 4.4. This step provides real-time, updated marking of specular reflection areas, ensuring that distorted pixels are correctly masked during multi-source fusion and that the remaining information is utilized according to weights.

[0152] In step S140 , the dynamic mask is fused based on the multimodal sensing data to obtain a fused feature map, and an event list is identified according to a preset event template.

[0153] The event list includes the event type and the event three-dimensional position.

[0154] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S140 specifically includes the following steps:

[0155] In step S141 , each pixel in the multimodal sensing data located in a strong reflection area is subjected to weighted preprocessing in accordance with weight attenuation, combined with a dynamic mask and a pixel weight map, to obtain weighted multimodal data.

[0156] In step 142, weighted fusion is performed on the weighted multimodal data according to a preset ratio to obtain a fused feature map, and local features are extracted from the fused feature map through a convolutional feature extraction network to obtain a local feature map.

[0157] Step S143 : Calculate the window cosine similarity based on each preset event template in the preset event template library, and retain the event templates that meet the candidate conditions as candidate templates to obtain a candidate event list.

[0158] In step S144 , each template index in the candidate event list is mapped to a specific event type, and a final event list is identified by combining the timestamp and the three-dimensional position of the event.

[0159] In a specific embodiment, the present invention provides an intelligent vehicle emergency response method for sudden situations inside and outside the vehicle, step 5, multi-source information fusion and event recognition.

[0160] The following steps are involved:

[0161] Step 5.1, mask weighted preprocessing.

[0162] Specifically, for each pixel, if it is a highly reflective area (determined by RGB intensity), the value is attenuated according to the weight, while non-reflective areas retain the original value. The final output is a weighted effective image, a weighted depth map, and a weighted echo map. This step weakens the data impact by weighting the distorted pixel areas, ensuring that false information interference is reduced during subsequent fusion, while retaining some potentially useful features.

[0163] Step 5.2: Cross-domain feature vector fusion.

[0164] Specifically, at each pixel, the weighted effective image, weighted depth map, and weighted echo map obtained in step 5.1 are weighted proportionally to synthesize the fusion features, so as to map the three-source data into a unified calibration space, taking into account visual texture, geometric distance, and acoustic reflection, and providing a one-dimensional fusion map for template matching.

[0165] Step 5.3: Feature extraction and scene template matching.

[0166] Specifically, a convolutional feature extraction network (shallow convolution + ReLU + max pooling) is applied to the fused features obtained in step 5.2 to generate a feature map. Next, the windowed cosine similarity is calculated for each template in the preset event template library. Templates whose cosine similarity reaches a set threshold (e.g., 0.7) are retained as candidate templates to generate a list of candidate events. This step quickly locates the area on the fused feature map that best matches the preset emergency event pattern, preliminarily identifying the possible event type.

[0167] Step 5.4: Spatiotemporal coherence verification and location clustering.

[0168] Specifically, the similarity of the first n frames of the same template in the candidate event list obtained in step 5.3 is examined. If it is greater than 0.5, the corresponding event is considered to persist. DBSCAN clustering is performed on the 3D coordinates of all events within the frame to remove isolated false positives. Finally, the center of gravity of the largest cluster is used as the final location. This step utilizes temporal and spatial consistency to filter out occasional false positives, improving the stability and confidence of event recognition while accurately outputting 3D location coordinates.

[0169] Step 5.5: Output event tags and trigger subsequent responses.

[0170] Specifically, each template index is mapped to a specific event type, such as "pedestrian intrusion", "obstacle ahead", "abnormal occupant in the vehicle", etc., and a message is formed with the timestamp, event type, and three-dimensional position of the event. The message is published to the emergency decision-making unit through the vehicle bus or ROS topic to trigger the generation of intelligent driving instructions.

[0171] Step S150 , in response to the event list, an emergency response strategy is retrieved according to a priority sequence, and corresponding intelligent driving instructions are executed according to the emergency response strategy.

[0172] In some embodiments, the intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle provided by the present invention, step S150 specifically includes the following steps:

[0173] Step S151 : Calculate the priority score of each event in the event list, and sort the events in the event list according to the priority score, so as to retrieve the policy template corresponding to each event in the sorted event list.

[0174] Step S152: Calculate the vehicle speed, driving angle, and instruction execution time according to the strategy template, and generate intelligent driving instructions based on the vehicle speed, driving angle, and instruction execution time.

[0175] Step S153: Send the intelligent driving instruction to the intelligent driving vehicle terminal through the CAN bus, and obtain the vehicle driving status in real time when the intelligent driving vehicle executes the intelligent driving instruction.

[0176] In a specific embodiment, the present invention provides an intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle, step 6, emergency response strategy generation and execution.

[0177] The following steps are involved:

[0178] Step 6.1, policy priority evaluation.

[0179] Specifically, first, for each event in the event list finally filtered out in step 5, calculate its corresponding priority score , the expression is:

[0180]

[0181] Where, Represents the event type weight constant (e.g. pedestrian intrusion = 10, obstacle = 8); is the Euclidean distance from the vehicle's current position to the event; ∈ = 0.1 is a constant to prevent division by 0.

[0182] In this embodiment, after calculating the priority score of each event, the events are sorted in descending order of score to obtain a priority list. This step quantifies the urgency of each emergency, giving priority to the most critical scenarios and ensuring that the closest and most dangerous targets are avoided first.

[0183] Step 6.2, optimal strategy retrieval.

[0184] Specifically, for each event in the priority list obtained in step 6.1, the corresponding policy template is retrieved from the policy library. For example, pedestrian intrusion corresponds to emergency braking + assisted by steering fine-tuning. Then, the top 2-3 policies are selected in order of score, and a set of selected policies corresponding to the priority is constructed. This can quickly obtain an executable plan based on the event type and priority.

[0185] Step 6.3: Chassis control command generation.

[0186] Specifically, according to the selected strategy set generated in step 6.2, the speed reduction, driving direction adjustment angle, and instruction execution time required for each strategy are calculated, and the speed reduction, driving direction adjustment angle, and instruction execution time required for each strategy are combined into intelligent driving instructions, which are sorted in the original priority order to obtain an instruction sequence.

[0187] Step 6.4: Instruction issuance and execution feedback.

[0188] Specifically, the command sequence generated in step 6.3 is sent to the actuator and brake controller one by one through the CAN bus or vehicle Ethernet, and execution feedback is received in real time when the command is executed. If the deviation between the feedback result and the expected result exceeds the set threshold, the command or fault event type is immediately switched to ensure that the strategy is actually implemented.

[0189] The following describes the intelligent driving vehicle emergency response device for emergencies inside and outside the vehicle provided by the present invention. The intelligent driving vehicle emergency response device for emergencies inside and outside the vehicle described below and the intelligent driving vehicle emergency response method for emergencies inside and outside the vehicle described above can be referenced to each other.

[0190] like Figure 2 As shown, in one embodiment, an intelligent driving vehicle emergency response device for sudden situations inside and outside the vehicle includes a polarization monitoring module, a mirror glare reduction module, a data calibration module, a feature fusion and event recognition module, and an instruction issuing module.

[0191] The polarization monitoring module is used to obtain the polarization intensity of the external glass curtain wall of the intelligent vehicle at different reflection angles in real time, and generate a polarization state distribution diagram based on the polarization intensity.

[0192] The specular glare reduction module is used to calibrate the polarization intensity of different reflection intensities and incident angles based on the polarization state distribution diagram through the adaptive polarization filter component to reduce the specular glare component.

[0193] The data calibration module is used to use multiple sensors to synchronously collect multimodal sensing data in the mirror reflection failure area, and to repair and compensate the distorted area in the optical image to generate a dynamic mask in the mirror reflection area.

[0194] The feature fusion and event recognition module is used to fuse the dynamic mask based on multimodal sensing data to obtain a fused feature map and identify an event list according to a preset event template.

[0195] The instruction issuing module is used to retrieve the emergency response strategy according to the priority sequence in response to the event list, and execute the corresponding intelligent driving instructions according to the emergency response strategy.

[0196] Among them, multimodal sensing data is divided into optical images, radar point clouds and ultrasonic echoes, and the event list contains event type and event three-dimensional position.

[0197] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of such features. "Multiple" means two or more, unless otherwise specifically defined.

[0198] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0199] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0200] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0201] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0202] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (not exhaustive) of computer-readable media include: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0203] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0204] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0205] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0206] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A smart driving vehicle emergency response method for sudden situations inside and outside the vehicle, characterized by: The method comprises: Acquire the polarization intensity of the glass curtain wall outside the intelligent vehicle at different reflection angles in real time, and generate a polarization state distribution diagram based on the polarization intensity; Calibrate the polarization intensity of different reflection intensities and incident angles based on the polarization state distribution diagram through an adaptive polarization filter component to reduce the specular glare component; Multiple sensors are used to synchronously collect multimodal sensing data in the mirror reflection failure area, and the distorted areas in the optical image are repaired and compensated to generate a dynamic mask in the mirror reflection area. Based on the multimodal sensing data, the dynamic mask is fused to obtain a fused feature map, and an event list is identified according to a preset event template; Retrieving an emergency response strategy in a priority sequence in response to the event list, and executing a corresponding intelligent driving instruction according to the emergency response strategy; The multimodal sensing data includes optical images, radar point clouds, and ultrasonic echoes, and the event list includes event types and event three-dimensional positions; The method of acquiring the polarization intensity of the outer glass curtain wall of the intelligent vehicle at different reflection angles in real time and generating a polarization state distribution diagram based on the polarization intensity includes: Obtaining a sampling timestamp sequence of the camera, and controlling the polarization sensor to resample by linear interpolation according to the sampling timestamp sequence to obtain a unified timing sequence and multi-angle light intensity after time alignment; Collecting a pixel-level light intensity array aligned with the camera through a plurality of polarization filters and an imaging array on the polarization sensor, and constructing a polarization intensity matrix based on the pixel-level light intensity array; Calculating the polarization angle corresponding to each pixel position in the polarization intensity matrix by Stokes component difference to construct a polarization angle matrix; Performing pseudo-color mapping on the polarization angle matrix and adjusting brightness based on the sum of multi-angle illumination intensities at each pixel position to generate the polarization state distribution map; The method of calibrating the polarization intensity of different reflection intensities and incident angles based on the polarization state distribution diagram by the adaptive polarization filter component to reduce the specular glare component includes: Performing a weighted average of the polarization angle matrix and the sum of the multi-angle light intensity at each pixel position to determine the global optimal filtering angle, and selecting a wave plate according to the current dispersion degree and reflection spectrum width to obtain a filtering phase delay value; Determining filter parameters based on the global optimal filtering angle and the filtering phase delay value, and sending a rotation instruction to the stepper motor through the single-chip microcomputer to control the filter to rotate to the global optimal filtering angle; Output voltage to the liquid crystal delay unit through the DAC to achieve the filtering phase delay value, and return the corresponding actual filtering angle and actual filtering phase delay value in real time through the encoder and voltage sensor; The adjusted polarization state distribution diagram, light intensity matrix and reflection mask are obtained according to the actual filtering angle and the actual filtering phase delay value, the light intensity ratio of the mask area to the full field of view is calculated, and the residual glare ratio is output.

2. The intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle according to claim 1 is characterized in that: The method comprises: utilizing multiple sensors to synchronously collect multimodal sensing data in the mirror reflection failure area, and repairing and compensating the distorted area in the optical image to generate a dynamic mask in the mirror reflection area. The method comprises: The microcontroller monitors the start time of the calibrated unified timing sequence, sends a standardized trigger frame containing a timestamp field on the CAN bus, and outputs a bus trigger instruction; The bus trigger instruction is sent to the camera. The camera performs sampling with fixed exposure parameters in response to the bus trigger instruction and reads the pixel array using a global shutter mode to obtain an optical image frame.

3. The intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle according to claim 2 is characterized in that: The method further comprises: utilizing multiple sensors to synchronously collect multimodal sensing data in the mirror reflection failure area, and repairing and compensating the distorted area in the optical image to generate a dynamic mask in the mirror reflection area. The bus trigger instruction is sent to the millimeter wave radar unit. The millimeter wave radar unit transmits an FMCW detection signal at a fixed pulse frame rate in response to the bus trigger instruction, and samples the echo on the receiving array to generate a point cloud matrix. Sending the bus trigger instruction to the ultrasound unit, the ultrasound unit sending pulses along a fixed frequency in response to the bus trigger instruction and collecting echo samples on the earpiece array to generate an echo sequence; The central fusion unit receives the point cloud matrix and echo sequence from the millimeter wave radar unit and the ultrasonic unit, and fuses and encapsulates the point cloud matrix, echo sequence and optical image based on the trigger timestamp to obtain a multimodal data packet.

4. The intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle according to claim 3 is characterized in that: The method further comprises: utilizing multiple sensors to synchronously collect multimodal sensing data in the mirror reflection failure area, and repairing and compensating the distorted area in the optical image to generate a dynamic mask in the mirror reflection area. The camera and radar are calibrated with external parameters using a calibration plate to obtain the rotation matrix and translation vector between the camera and radar, and the radar point cloud is mapped to the pixel coordinate system. Obtain the minimum depth value of all valid mapping points, combine bilinear interpolation to generate a depth map, and project the ultrasonic echo intensity to the corresponding pixel position according to the rigid connection parameters between the ultrasonic sensor and the camera to obtain an echo intensity map; Based on the depth map and the echo intensity map, the reflectance index corresponding to each pixel coordinate is calculated in combination with the Stokes component difference matrix, and the adaptive threshold is calculated according to the full-image statistical characteristics of the reflectance index; Each pixel is subjected to binary processing to obtain a binary mask, and the binary mask is smoothed by using a median filter to generate the dynamic mask and the corresponding pixel weight map.

5. The intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle according to claim 4 is characterized in that: The method of fusing the dynamic mask based on the multimodal sensing data to obtain a fused feature map and identifying an event list according to a preset event template includes: Performing weighted preprocessing on each pixel in the multimodal sensing data located in a strong reflection area according to weight attenuation, combining a dynamic mask and a pixel weight map, to obtain weighted multimodal data; Performing weighted fusion on the weighted multimodal data according to a preset ratio to obtain a fused feature map, and performing local feature extraction on the fused feature map through a convolutional feature extraction network to obtain a local feature map; Calculate the window cosine similarity based on each preset event template in the preset event template library, and retain the event templates that meet the candidate conditions as candidate templates to obtain a candidate event list; Each template index in the candidate event list is mapped to a specific event type, and the final event list is identified by combining the timestamp and the three-dimensional position of the event.

6. The intelligent driving vehicle emergency response method for sudden situations inside and outside the vehicle according to claim 1 is characterized in that: The step of retrieving an emergency response strategy according to a priority sequence in response to the event list and executing a corresponding intelligent driving instruction according to the emergency response strategy includes: Calculating a priority score for each event in the event list, and sorting the events in the event list according to the priority scores, so as to retrieve a policy template corresponding to each event in the sorted event list; Calculating the vehicle speed, driving angle, and instruction execution time according to the strategy template, and generating the intelligent driving instruction based on the vehicle speed, driving angle, and instruction execution time; The intelligent driving instruction is sent to the intelligent driving vehicle terminal through the CAN bus, and the vehicle driving status is obtained in real time when the intelligent driving vehicle executes the intelligent driving instruction.

7. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images