Automobile emergency door and window opening system based on artificial intelligence

Abstract

The invention discloses an automobile emergency door and window opening system based on artificial intelligence, and the system comprises an image collection module which generates an image frame sequence of a driver, passengers and an automobile window area; the sensor acquisition module is used for acquiring acceleration, sound intensity and door lock and window control states and constructing time synchronization data; the improved spatial propagation network construction module comprises a residual-driven two-way time regulation and control module, an event atlas matching gating module, a structure-maintained region propagation module and a confidence-guided anomaly repair module; the four modules respectively generate a time-sensitive spatial feature graph, a gating tensor, a regional topological graph, an abnormal scoring graph and a boundary confidence graph; and the instruction generation module fuses the characteristics and outputs door opening and window opening instructions. According to the invention, the intelligent processing capability of the automobile to deal with emergency situations is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent automobile control, and particularly relates to an emergency door and window opening system for an automobile based on artificial intelligence. BACKGROUND

[0002] Under the background of the continuous development of current intelligent automobiles, the vehicle control system plays an increasingly important role in ensuring the safety of passengers, especially when the vehicle encounters emergency situations such as collision, water falling, high temperature or smoke, whether the doors and windows can be automatically and timely opened becomes a key factor affecting the survival probability of the passengers in the vehicle. For this reason, the industry has proposed various automatic door and window opening systems, some of which trigger by setting sensor thresholds, for example, when the collision intensity exceeds the set value, the preset opening instruction is immediately executed. Although this kind of method has a simple implementation mechanism, it often lacks overall perception of the in-vehicle environment and the state of the passengers, is prone to misjudgment or response delay, and is difficult to cope with complex and variable actual scenarios.

[0003] In recent years, with the maturity of computer vision technology, some researches have begun to introduce image recognition methods for analyzing in-vehicle pictures and recognizing passenger behavior to assist decision-making. However, most of the methods mainly rely on static images for target detection or classification, and cannot effectively utilize the time sequence structure information in image sequences, making it difficult to capture rapid changes occurring in the scene. In addition, sensor data and image data are usually processed separately, and a unified data fusion framework has not been established, which makes it difficult for the system to achieve accurate judgment and rapid response when responding to sudden events.

[0004] Existing spatial propagation network models have made certain progress in some image tasks, such as image-guided filling or depth estimation, but there are still limitations when directly used for emergency control tasks. On the one hand, traditional models lack the ability to model time series and cannot reflect the rules of structural changes between image frames. On the other hand, the spatial propagation process of the traditional model is relatively rigid and difficult to adapt to the complex and dynamic semantic structure in the vehicle. At the same time, the key abnormal information provided by the sensor cannot be effectively guided and utilized in the propagation process, resulting in insufficient robustness of the system in abnormal state recognition and response.

[0005] Therefore, how to provide an emergency door and window opening system for an automobile based on artificial intelligence is a problem that those skilled in the art need to solve. SUMMARY

[0006] One purpose of the present application is to propose an artificial intelligence-based automobile emergency door and window opening system, which fuses vehicle-mounted image sequences and sensor data to construct a spatial propagation network containing residual-driven bidirectional time regulation, event atlas matching gating, structure-preserving regional propagation and confidence-guided anomaly repair, and describes in detail the whole process of automatically identifying emergency situations and generating door and window opening control instructions in emergency situations, with the advantages of fast response speed, high identification accuracy and strong multi-source information collaborative processing capability.

[0007] The artificial intelligence-based automobile emergency door and window opening system according to the embodiment of the present application comprises:

[0008] An image acquisition module is configured to acquire driving area images, occupant area images and window area images, perform image decoding and frame separation processing, and generate image frame sequences.

[0009] A sensor acquisition module is configured to acquire instantaneous acceleration values of an acceleration sensor, sound intensity values of a sound pickup, and door lock state signals and window control state signals recorded by a vehicle-mounted control system, align based on timestamps and image frame sequences, and construct a time-synchronized data input structure.

[0010] An improved spatial propagation network construction module is configured to construct an improved spatial propagation network containing a residual-driven bidirectional time regulation module, an event atlas matching gating module, a structure-preserving regional propagation module and a confidence-guided anomaly repair module.

[0011] The residual-driven bidirectional time regulation module is configured to extract structural residuals between image frames, calculate forward and backward propagation weights, and generate time-sensitive spatial feature maps.

[0012] The event atlas matching gating module is configured to construct an event graph based on time-synchronized sensor data, and perform graph structure matching with the spatial feature maps to generate a gating tensor.

[0013] The structure-preserving regional propagation module is configured to divide semantic regions, construct a regional topology graph, and adjust propagation directions and connection relationships according to geometric constraints.

[0014] The confidence-guided anomaly repair module is configured to construct an anomaly score graph and a boundary confidence graph based on structural residuals, perform reverse multi-scale propagation in the score super-threshold region, and limit the boundary propagation range.

[0015] An instruction generation module is configured to fuse the time-sensitive spatial feature maps, the gating tensor, the regional topology graph, the anomaly score graph and the boundary confidence graph, generate a fused propagation feature map, calculate a state score result, generate door opening instructions and window opening instructions according to the score result, and output to the execution units of the door control system and the window control system.

[0016] Optionally, the modules are realized through the following method:

[0017] S1, collect image frame sequence and sensor data, the image frame sequence includes driving area, passenger area and window area image, the sensor data includes acceleration, sound intensity and door and window state signal;

[0018] S2, construct an improved space propagation network, including a residual-driven bidirectional time regulation module, an event graph matching gating module, a structure-preserving regional propagation module and a confidence-guided anomaly repair module;

[0019] S3, extract the structural residual between image frames through the residual-driven bidirectional time regulation module, adjust the forward and backward propagation weights, and generate time-sensitive spatial feature maps;

[0020] S4, through the event graph matching gating module, the sensor data is constructed into an event graph, and the spatial feature map is matched with the graph structure to generate a gating tensor;

[0021] S5, through the structure-preserving regional propagation module, the semantic region is divided, the regional topology graph is constructed, and the propagation direction and connection relationship are adjusted according to the geometric constraint;

[0022] S6, through the confidence-guided anomaly repair module, an anomaly score map and a boundary confidence map are generated, reverse multi-scale propagation is performed in the threshold region, and the boundary propagation range is limited;

[0023] S7, fuse the outputs of the four modules to generate a fused propagation feature map, input the emergency state discrimination module, and generate a recognition result, output the open door instruction and the open window instruction to the execution unit when the trigger condition is met.

[0024] Optionally, the image frame sequence is generated by image decoding and frame separation processing of the vehicle-mounted video stream, the sensor data includes instantaneous acceleration value output by the acceleration sensor, sound intensity value collected by the sound pickup, and door lock state and window control state signals recorded by the vehicle-mounted control system, and the image frame sequence and the sensor data are aligned by time stamp to construct a time-synchronized data input structure.

[0025] Optionally, S3 specifically includes:

[0026] S31, extract the current image frame, the previous image frame and the next image frame from the image frame sequence, and input them into the feature extraction network composed of convolution unit, batch normalization unit and channel attention unit respectively, to generate current image frame feature map, previous image frame feature map and next image frame feature map;

[0027] S32, calculate the structural residual map R f(x, y), and a structure residual map R between the previous image frame and the current image frame b (x, y), the structure residual map is calculated in a cosine similarity guided manner:

[0028]

[0029] wherein R(x, y) represents a pixel value of the structure residual map at position (x, y), represents a feature value of the previous image frame or the subsequent image frame at position (x, y) on a channel with channel index c, represents a feature value of the current image frame at position (x, y) on a channel with channel index c, and C represents a number of channels of the image feature map;

[0030] S33, performing an average pooling operation on the structure residual map R f (x, y) and R b (x, y) to generate a forward response intensity and a backward response intensity respectively, and calculating a forward propagation weight and a backward propagation weight according to the response intensities;

[0031] S34, applying the corresponding propagation weights to the previous image frame feature map and the subsequent image frame feature map respectively and performing weighted fusion to generate a time fusion feature map, and concatenating the time fusion feature map with the current image frame feature map in a channel dimension to form a combined feature map;

[0032] S35, performing an average pooling operation on the structure residual map R f (x, y) and R b (x, y) to generate a spatial attention map, and applying spatial weighting to the combined feature map to generate a residual regulation feature map;

[0033] S36, inputting the residual regulation feature map into a channel attention module to generate a time-sensitive spatial feature map.

[0034] Optionally, the S4 specifically comprises:

[0035] S41, constructing an event graph based on the time-synchronized sensor data, the event graph being composed of an event node set and an edge set, an event node representing a state of acceleration, sound intensity and door / window state signal at a certain time point, and the edge set representing an order relationship between adjacent time point states;

[0036] S42, encoding a spatial feature map of a current frame in the image frame sequence into a feature vector set, each spatial position corresponding to a feature vector;

[0037] S43, constructing a graph mapping tensor M(i, j) according to structural similarity between the spatial feature vector and the event node embedding:

[0038]

[0039] where M(i,j) denotes the matching weight between the i-th spatial feature location and the j-th event node, s i denotes the feature vector of the i-th spatial location, e j denotes the embedding representation of the j-th event node, γ denotes a scaling factor, and cos(·,·) denotes the cosine similarity;

[0040] S44, input the graph mapping tensor M(i,j) into a gating structure, and generate a gating tensor for depicting the response distribution of the event graph on the spatial structure in combination with the current frame spatial feature map.

[0041] Optionally, the S44 specifically includes:

[0042] S441, the gating structure is a module composed of a graph response generation unit, a feature fusion unit and an adjustment calculation unit, configured to receive the graph mapping tensor and the spatial feature map, and establish a response relationship between the event graph node and the graph mapping tensor at each image spatial location;

[0043] S442, the graph response generation unit calculates the graph response result of each spatial location based on the graph mapping tensor and the embedding representation of the event graph node;

[0044] S443, the feature fusion unit splices the graph response result and the spatial feature map vector of the corresponding spatial location to form a joint feature representation;

[0045] S444, the adjustment calculation unit performs linear transformation and activation function calculation on the joint feature representation to generate a gating response value as an adjustment coefficient of the spatial location;

[0046] S445, map the gating response value to the channel dimension of the spatial feature map, and perform weighted processing on the image feature vector of each spatial location to output a gating tensor, which is used to adjust the response intensity distribution of the image spatial location in the structure propagation network.

[0047] Optionally, the S5 specifically includes:

[0048] S51, input the spatial location and channel dimension feature of each pixel in the spatial feature map, divide semantic regions by using feature similarity and spatial connectivity, and the semantic region is a two-dimensional sub-region set composed of continuous pixel locations;

[0049] S52, construct a region topology graph in each semantic region, the nodes of the region topology graph represent pixel location indexes, and the edges represent connections between nodes having spatial contact relationship;

[0050] S53, extract the main direction vector inside the semantic region, calculate the included angle between the spatial direction vector connected by each edge and the main direction, and generate a direction consistency score matrix;

[0051] S54, set an included angle threshold, judge whether the edge connection direction meets the consistency condition with the main direction, and remove the edge connection that does not meet the condition;

[0052] S55, update the edge connection direction according to the direction consistency judgment result, and generate a region topology graph after structure adjustment.

[0053] Optionally, the S6 specifically includes:

[0054] S61, receive a structure residual result, calculate an abnormal score map based on the residual amplitude between pixels, and each pixel position in the abnormal score map corresponds to an abnormal score value;

[0055] S62, set an abnormal score threshold τ, mark the pixel position with a score value greater than τ as an abnormal region, and generate an abnormal mask map;

[0056] S63, extract the abnormal region boundary information in the abnormal mask map, generate a boundary confidence map based on the image gradient, boundary closure degree and path continuity, and define the boundary confidence value:

[0057] β i =α1·g i +α2·c i +α3·p i ;

[0058] Wherein, β i is the confidence value of pixel i in the boundary confidence map, g i is the image gradient value, c i is the boundary closure degree index, p i is the propagation path continuity score, and α1, α2 and α3 are weighting coefficients, satisfying α1+α2+α3=1;

[0059] S64, select the position with the maximum score value in the abnormal score map as the propagation starting point, and execute the reverse multi-scale propagation operation combined with the boundary confidence map;

[0060] S65, limit the propagation path in each scale by boundary, skip the position with a boundary confidence value lower than the lower limit, and output the abnormal repair map after propagation adjustment.

[0061] Optionally, the S7 specifically includes:

[0062] S71, the time-sensitive spatial feature map generated by the fusion residual-driven bidirectional time regulation module, the gating tensor generated by the event map matching gating module, the regional topology map generated by the structure-preserving region propagation module, and the abnormal score map and the boundary confidence map generated by the confidence-guided abnormal repair module are aligned according to the image frame index and spliced in the channel dimension to generate a fusion propagation feature map;

[0063] S72, performing feature extraction and vector mapping operations on the fusion propagation feature map to generate a state response vector;

[0064] S73, setting a trigger threshold, weighting and calculating each component in the state response vector to obtain a score value, and comparing the score value with the trigger threshold to determine whether the instruction generation condition is met;

[0065] S74, when the score value meets the instruction generation condition, generating an open door instruction and an open window instruction, and sending them to the execution units of the vehicle door control system and the vehicle window control system respectively.

[0066] The beneficial effects of the present application are:

[0067] The present application proposes an artificial intelligence-based emergency door and window opening system for a vehicle, focusing on improving the response speed and intelligent processing capability of the vehicle in emergency situations. The system integrates vehicle image frame sequences and various sensor data, and realizes the deep linkage of image and event information by constructing an improved spatial propagation network, breaking the limitations of traditional state judgment relying on a single data source, and effectively improving the recognition accuracy of the emergency state in the vehicle and the timeliness of the control response.

[0068] In terms of method design, the system constructs a time-sensitive spatial representation based on the residual features between image frames, and constructs a graph structure based on sensor events, and realizes the cooperative propagation of event information and image space through a gating mechanism. Further, by maintaining the regional topological relationship and optimizing the propagation direction, the local structural information is stably transmitted in the complex vehicle environment; at the same time, the abnormal score and boundary confidence mechanism is introduced, which can identify the emergency state and effectively repair and limit the possible incomplete or abnormal region.

[0069] Finally, the system fuses the multi-dimensional features of the above modules, and after identifying the abnormal state and meeting the preset judgment condition, it can automatically generate an open door and window control instruction to release the passenger escape channel in time. The overall scheme has compact logic and clear structure, and strengthens the intelligent processing capability of the vehicle-mounted system in emergency environment, providing a practical technical path for improving the vehicle safety level and passenger survival opportunity. BRIEF DESCRIPTION OF DRAWINGS

[0070] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are meant to explain the present application but are not intended to limit the application. In the drawings:

[0071] Fig. 1 Flow chart of the artificial intelligence-based automobile emergency door and window opening system proposed in the present application;

[0072] Fig. 2 Schematic diagram of the space propagation network structure of the artificial intelligence-based automobile emergency door and window opening system proposed in the present application;

[0073] Fig. 3 Fusion feature generation and instruction output flow chart of the artificial intelligence-based automobile emergency door and window opening system proposed in the present application. DETAILED DESCRIPTION

[0074] The present application will now be described in further detail with reference to the drawings. These drawings are simplified schematic diagrams that only schematically show the basic structure of the present application and thus only show the components relevant to the present application.

[0075] Reference Figs. 1-3 The artificial intelligence-based automobile emergency door and window opening system comprises:

[0076] An image acquisition module is configured to acquire images of a driving area, an occupant area and a window area, perform image decoding and frame separation processing, and generate image frame sequences.

[0077] A sensor acquisition module is configured to acquire instantaneous acceleration values of an acceleration sensor, sound intensity values of a sound pickup, and door lock state signals and window control state signals recorded by a vehicle-mounted control system, align based on timestamps and image frame sequences, and construct a time-synchronized data input structure.

[0078] An improved space propagation network construction module is configured to construct an improved space propagation network comprising a residual-driven bidirectional time regulation module, an event graph matching gate module, a structure-preserving regional propagation module and a confidence-guided anomaly repair module.

[0079] The residual-driven bidirectional time regulation module is configured to extract structural residuals between image frames, calculate forward and backward propagation weights, and generate time-sensitive spatial feature maps.

[0080] The event graph matching gate module is configured to construct an event graph based on time-synchronized sensor data and perform graph structure matching with the spatial feature maps to generate a gate tensor.

[0081] The structure-preserving regional propagation module is configured to divide semantic regions, construct regional topological graphs, and adjust propagation directions and connection relationships according to geometric constraints.

[0082] an abnormality repair module based on confidence guidance, configured to construct an abnormal score map and a boundary confidence map based on the structural residual, perform reverse multi-scale propagation in a score-over-threshold region, and limit the boundary propagation range;

[0083] an instruction generation module, configured to fuse the time-sensitive spatial feature map, the gating tensor, the region topology map, the abnormal score map and the boundary confidence map, generate a fused propagation feature map, calculate a state score result, generate an opening door instruction and an opening window instruction according to the score result, and output the instructions to an execution unit of a vehicle door control system and a vehicle window control system.

[0084] The application realizes real-time synchronous acquisition of image information and key state data of different regions in the vehicle through the cooperative work of the image acquisition module and the sensor acquisition module, and constructs a unified data input structure. The four functional modules contained in the improved spatial propagation network cooperate at the structure and function level, can cooperatively model the image time sequence features, event state information, spatial semantic structure and abnormal interference factors, and provide accurate decision basis for subsequent control instruction generation.

[0085] In the embodiment, the modules are realized through the following methods:

[0086] S1, acquiring image frame sequences and sensor data, the image frame sequences including images of a driving region, a passenger region and a window region, and the sensor data including acceleration, sound intensity and door / window state signals;

[0087] S2, constructing an improved spatial propagation network, including a residual-driven bidirectional time regulation module, an event atlas matching gating module, a structure-preserving region propagation module and an abnormality repair module based on confidence guidance;

[0088] S3, extracting the structural residual between image frames through the residual-driven bidirectional time regulation module, adjusting the forward and backward propagation weights, and generating a time-sensitive spatial feature map;

[0089] S4, constructing an event graph from the sensor data through the event atlas matching gating module, performing graph structure matching with the spatial feature map, and generating a gating tensor;

[0090] S5, dividing semantic regions through the structure-preserving region propagation module, constructing a region topology map, and adjusting the propagation direction and connection relationship according to geometric constraints;

[0091] S6, generating an abnormal score map and a boundary confidence map through the abnormality repair module based on confidence guidance, performing reverse multi-scale propagation in a score-over-threshold region, and limiting the boundary propagation range;

[0092] S7, fusing the outputs of the four modules to generate a fused propagation feature map, inputting the fused propagation feature map into an emergency state discrimination module to generate a discrimination result, and outputting a door opening instruction and a window opening instruction to an execution unit when a triggering condition is met.

[0093] The application adopts a structured method to process information input of different dimensions in stages, respectively performs feature extraction and fusion in the time sequence dimension, the spatial dimension, the semantic dimension and the abnormal response dimension, and finally outputs the door opening and window opening instructions through decision logic. The overall process is closely linked, so that the system has real-time response capability and high fault tolerance, and meets the intelligent control needs of the safety protection of the passengers in the vehicle in sudden dangerous events.

[0094] In the embodiment, the image frame sequence is generated by image decoding and frame separation processing of the vehicle-mounted video stream, the sensor data includes instantaneous acceleration values output by an acceleration sensor, sound intensity values collected by a sound pickup, and door lock state and window control state signals recorded by a vehicle-mounted control system, and the image frame sequence and the sensor data are aligned by timestamps to construct a time-synchronized data input structure.

[0095] The application constructs a time-synchronized data input structure based on the vehicle-mounted video stream and the sensor raw signals, effectively solves the error interference problem caused by asynchronous sampling. The image frames and the signals such as acceleration, sound intensity, door lock state and window control state are accurately aligned by timestamps, which provides a unified basis for subsequent event graph construction and multi-modal fusion, and improves the accuracy and stability of the system as a whole.

[0096] In the embodiment, the S3 specifically includes:

[0097] S31, extracting a current image frame, a previous image frame and a next image frame from the image frame sequence, and inputting them into a feature extraction network composed of a convolution unit, a batch normalization unit and a channel attention unit to generate a current image frame feature map, a previous image frame feature map and a next image frame feature map;

[0098] S32, calculating a structural residual map R f (x,y) between the previous image frame and the current image frame, and a structural residual map R b (x,y) between the next image frame and the current image frame at each spatial position (x,y), the structural residual map being calculated in a cosine similarity guided manner:

[0099]

[0100] wherein R(x,y) represents a pixel value of the structural residual map at position (x,y), represents a feature value of the previous image frame or the next image frame at position (x,y) in a channel with channel index c, represents the feature value of the current image frame at position (x, y) on the channel with channel index c, and C represents the number of channels of the image feature map;

[0101] S33, the structural residual map R f (x, y) and R b (x, y) and R f (x, y) and R b (x, y) and R

[0102] S34, respectively, the previous image frame feature map and the next image frame feature map are subjected to corresponding propagation weights and weighted fusion, a time fusion feature map is generated, and the current image frame feature map is spliced in the channel dimension to form a combined feature map;

[0103] S35, the structural residual map R f (x, y) and R b (x, y) and R f (x, y) and R b (x, y) and R

[0104] S36, the residual regulation feature map is input into a channel attention module to generate a time-sensitive spatial feature map.

[0105] The present application introduces a structural residual map, a propagation weight adjustment mechanism and an attention regulation method to establish a detailed process for extracting a time-sensitive spatial feature map from an image frame sequence. The temporal response relationship is jointly modeled by using the residual between images and spatial attention, and the response strength of the key region of the image is further enhanced by channel attention, so that the system can accurately capture the small image changes related to the dangerous event.

[0106] In the embodiment, the S4 specifically comprises:

[0107] S41, based on the time-synchronized sensor data, an event graph is constructed, the event graph is composed of an event node set and an edge set, the event node represents the state of acceleration, sound intensity and door and window state signal at a certain time point, and the edge set represents the sequential relationship between adjacent time point states;

[0108] S42, the spatial feature map of the current frame in the image frame sequence is encoded into a feature vector set, and each spatial position corresponds to a feature vector;

[0109] S43, according to the structural similarity between the spatial feature vector and the event node embedding, a graph mapping tensor M(i, j) is constructed:

[0110]

[0111] wherein M(i,j) represents the matching weight between the i-th spatial feature position and the j-th event node, s i represents the feature vector of the i-th spatial position, e j represents the embedding representation of the j-th event node, and γ represents a scaling factor, and cos(·,·) represents the cosine similarity.

[0112] S44, input the graph mapping tensor M(i,j) into a gating structure, and generate a gating tensor in combination with the spatial feature map of the current frame, for describing the response distribution of the event graph on the spatial structure.

[0113] The application constructs a structural mapping relationship between the event graph and the image spatial feature, introduces a graph mapping tensor and a gating mechanism, and synchronously perceives the sensor state change and the image content.

[0114] In the embodiment, the S44 specifically comprises:

[0115] S441, the gating structure is a module composed of a graph response generation unit, a feature fusion unit and an adjustment calculation unit, for receiving the graph mapping tensor and the spatial feature map, and establishing a response relationship between the event graph node and the graph mapping tensor at each image spatial position;

[0116] S442, the graph response generation unit calculates the graph response result of each spatial position based on the embedding representation of the graph mapping tensor and the event graph node;

[0117] S443, the feature fusion unit splices the graph response result and the spatial feature map vector of the corresponding spatial position to form a joint feature representation;

[0118] S444, the adjustment calculation unit performs linear transformation and activation function calculation on the joint feature representation to generate a gating response value as an adjustment coefficient of the spatial position;

[0119] S445, the gating response value is mapped to the channel dimension of the spatial feature map, and the image feature vector of each spatial position is weighted processed to output a gating tensor, which is used to adjust the response intensity distribution of the image spatial position in the structure propagation network.

[0120] The application adopts a modular gating structure, guides the spatial feature map to generate a gating tensor based on the graph response result, and makes the response of the sensor signal in the image space more recognizable. The graph-response generation unit ensures the response consistency between the graph-image structure, and the feature fusion and adjustment calculation process further amplifies the region feature with high event matching degree in the image, effectively improving the selectivity and accuracy of the structure propagation network.

[0121] In this embodiment, the S5 specifically includes:

[0122] S51, taking the spatial position and channel dimension features of each pixel in the spatial feature map as input, and dividing the semantic region by feature similarity and spatial connectivity, where the semantic region is a set of two-dimensional sub-regions consisting of continuous pixel positions;

[0123] S52. Construct a regional topology map in each semantic region, where nodes of the regional topology map represent pixel position indexes, and edge connections represent connections with spatial contact relationships between nodes.

[0124] S53, extracting the main direction vector inside the semantic region, calculating the angle between the spatial direction vector connected by each edge and the main direction, and generating a direction consistency score matrix;

[0125] S54: Setting an angle threshold to determine whether the edge connection direction and the main direction meet the consistency condition, and removing edge connections that do not meet the condition;

[0126] S55. Update the edge connection direction based on the direction consistency judgment result to generate a regional topology map after structural adjustment.

[0127] The semantic region partitioning and regional topology map construction method designed in this paper clearly expresses the semantic structure in image space. By analyzing the matching of edge connection directions with the main direction and performing geometric corrections, the propagation path structure is optimized. The resulting regional topology map has clear propagation directionality and regional connectivity, providing fundamental support for semantic propagation in complex scenarios.

[0128] In this embodiment, S6 specifically includes:

[0129] S61, receiving the structural residual result, and calculating an anomaly score map based on the residual amplitude between pixels, where each pixel position in the anomaly score map corresponds to an anomaly score value;

[0130] S62: Set an abnormality score threshold τ, mark pixel positions with a score value greater than τ as abnormal areas, and generate an abnormality mask map;

[0131] S63. Extract the abnormal region boundary information from the abnormal mask image, generate a boundary confidence map based on the image gradient, boundary closure, and path continuity, and define the boundary confidence value:

[0132] β i =α1·g i +α2·c i +α3·p i ;

[0133] Among them, β iis the confidence value of pixel i in the boundary confidence map, g i is the image gradient value, c i is the boundary closure index, p i is the propagation path continuity score, and α1, α2, and α3 are weighting coefficients satisfying α1+α2+α3=1;

[0134] S64, selecting the position with the maximum score value in the anomaly score map as the propagation starting point, and performing a reverse multi-scale propagation operation in combination with the boundary confidence map;

[0135] S65, performing boundary restriction on the propagation path at each scale, skipping positions with a boundary confidence value below a lower limit, and outputting an anomaly repair map after propagation adjustment.

[0136] The present application calculates an anomaly score map and a boundary confidence map based on a structural residual map, and limits the anomaly propagation path through the confidence map, effectively suppressing the feature pollution problem caused by false propagation. The method of jointly modeling the anomaly score and the image boundary information improves the robustness of the system in a high interference scene, and makes the anomaly detection and repair operation more consistent with human visual cognitive rules.

[0137] In the embodiment, the S7 specifically comprises:

[0138] S71, fusing the time-sensitive spatial feature map generated by the residual-driven bidirectional time regulation module, the gating tensor generated by the event map matching gating module, the regional topological map generated by the structure-preserving regional propagation module, and the anomaly score map and the boundary confidence map generated by the confidence-guided anomaly repair module, aligning them according to the image frame index and concatenating them in the channel dimension to generate a fused propagation feature map;

[0139] S72, performing feature extraction and vector mapping operations on the fused propagation feature map to generate a state response vector;

[0140] S73, setting a trigger threshold, performing weighted calculation on each component in the state response vector to obtain a score value, and comparing the score value with the trigger threshold to determine whether the instruction generation condition is met;

[0141] S74, when the score value meets the instruction generation condition, generating an open door instruction and an open window instruction, and sending them to the execution units of the door control system and the window control system, respectively.

[0142] The present application fuses the output features of each module of the improved spatial propagation network after aligning them according to the image frame index, generates a high-dimensional fused propagation feature map, and ensures the spatio-temporal consistency of information fusion. Through the state response vector scoring and trigger judgment mechanism, the intelligent identification of the emergency state is realized, and the open door and open window instructions are directly outputted when the conditions are met, effectively shortening the reaction time and reducing the operation complexity, and providing an intelligent response mechanism for the safety of the passengers in the vehicle.

[0143] Example 1

[0144] To verify the feasibility of the application in implementation, the application is applied to a set of vehicle simulation platform integrated with video acquisition, sensor monitoring and intelligent control. The platform builds a complete in-vehicle environment simulation system, equipped with three camera modules corresponding to the driving area, passenger area and window area respectively, and integrates acceleration sensor, sound pickup and vehicle control bus interface. Through the embedded NPU (Neural Processing Unit) and ARM processor jointly deploying the improved spatial propagation network proposed by the application, local calculation and real-time response are realized without relying on external servers.

[0145] The system operation process is as follows: image frame sequences are collected by the vehicle-mounted camera system at a rate of 20 frames per second, and standard RGB frames are generated by parsing and generating; acceleration, sound intensity and door and window state signals are sampled at 200ms, and all data are processed by timestamp synchronization to generate time-consistent input structure. The input structure is sent to the improved spatial propagation network, which extracts image structure changes through the residual-driven bidirectional time control module in turn, then matches the event graph constructed by the sensor with the spatial feature graph to generate the gating tensor, and then adjusts the feature propagation direction according to the geometric structure by the region propagation module, finally performs confidence repair operation in the abnormal area to generate the fusion propagation feature graph as the decision basis.

[0146] In the evaluation stage, 20 simulation test scenes covering collision, rollover, abnormal sound, severe braking and false alarm are designed to verify the discrimination ability and response efficiency of the system in complex environment one by one, and Table 1 lists some typical test examples:

[0147] Table 1 Emergency event identification and response test data

[0148]

[0149]

[0150] Table 1 shows that when the acceleration is greater than 6.0m / s 2 , the sound intensity exceeds 100dB and the abnormal score area exceeds 30% area, the system stably triggers the open door and open window instructions, and the average response time is 171ms, which is much lower than the manual intervention time delay, meeting the requirements of in-vehicle emergency control response. Taking T03 as an example, in the high acceleration and noise background, the image residual graph labels that significant structural disturbance occurs in the lower left side of the passenger area, the gating tensor shows that the event graph and image feature are strongly matched, the region propagation module labels the key propagation path and generates a high-confidence abnormal area distribution under the guidance of the confidence mechanism, and finally correctly determines the high-risk state and sends the open window and unlock instructions.

[0151] Further compared with the traditional rule-based door lock control method in the experiment, the average false positive rate is as high as 22.5%, while the false positive rate of the system is controlled at 5.4%. In addition, in the false trigger control test, the system remains stable and does not output false instructions in an environment below the threshold, showing excellent robustness.

[0152] To show the linkage effect between the output results of the modules, the following are some feature maps and qualitative evaluation data of module results:

[0153] Table 2: Module output consistency evaluation sample

[0154]

[0155] As can be seen from Table 2, in all successfully identified emergency events, the residual intensity map generally has more than 10 hot spot regions, the average gated response is greater than 0.7, and the region propagation adjustment frequency is high. The anomaly repair module can accurately suppress the untrusted propagation points on most paths, indicating that the modules are effectively coupled at the structural level, forming a highly reliable joint discrimination mechanism.

[0156] The above test results show that the automobile emergency door and window opening system of the present application has stable sensing, discrimination and control capabilities in complex environments, especially in the case of severe disturbance in the vehicle and limited passengers, it can effectively assist in automatically releasing door and window control instructions, and significantly improve the safety protection level in the vehicle.

[0157] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art, according to the technical solution and inventive concept of the present application, can make equivalent replacements or changes within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. The artificial intelligence-based emergency door and window opening system for cars is characterized by: include: An image acquisition module is used to acquire images of the driving area, passenger area, and window area, perform image decoding and frame separation processing, and generate an image frame sequence; The sensor acquisition module is used to collect the instantaneous acceleration value of the accelerometer, the sound intensity value of the microphone, and the door lock status signal and window control status signal recorded by the vehicle control system. Based on the alignment of the timestamp and the image frame sequence, a time-synchronized data input structure is constructed; Improved spatial propagation network construction module, used to construct an improved spatial propagation network including a residual-driven bidirectional temporal control module, an event graph matching gating module, a structure-preserving regional propagation module, and a confidence-guided anomaly repair module; A residual-driven bidirectional temporal control module is used to extract structural residuals between image frames, calculate forward and backward propagation weights, and generate time-sensitive spatial feature maps; The event graph matching gating module is used to build an event graph based on time-synchronized sensor data and perform graph structure matching with the spatial feature graph to generate a gated tensor; The structure-preserving regional propagation module is used to divide semantic regions, construct regional topology maps, and adjust propagation directions and connectivity relationships based on geometric constraints; The confidence-guided anomaly repair module is used to construct anomaly score maps and boundary confidence maps based on structural residuals, perform reverse multi-scale propagation in the score-exceeding-threshold region, and limit the boundary propagation range; The instruction generation module is used to fuse the time-sensitive spatial feature map, gated tensor, regional topology map, anomaly score map and boundary confidence map to generate a fused propagation feature map, calculate the state score result, generate door opening instructions and window opening instructions based on the score results, and output them to the execution units of the door control system and window control system.

2. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 1 is characterized in that: The modules are implemented as follows: S1. Acquire image frame sequences and sensor data. The image frame sequences include images of the driving area, passenger area, and window area. The sensor data includes acceleration, sound intensity, and door and window status signals. S2. Build an improved spatial propagation network, which includes a residual-driven bidirectional temporal control module, an event graph matching gating module, a structure-preserving regional propagation module, and a confidence-guided anomaly repair module. S3, extracting the inter-frame structural residuals of the image through the residual-driven bidirectional temporal control module, adjusting the forward and backward propagation weights, and generating a time-sensitive spatial feature map; S4, through the event graph matching gating module, the sensor data is constructed into an event graph, and the graph structure is matched with the spatial feature graph to generate a gating tensor; S5. Divide semantic regions through the structure-preserving regional propagation module, construct regional topology maps, and adjust propagation directions and connection relationships according to geometric constraints; S6. Generate an anomaly score map and a boundary confidence map through the confidence-guided anomaly repair module, perform reverse multi-scale propagation in the score exceeding threshold area, and limit the boundary propagation range; S7. Fusion of the outputs of the four modules generates a fusion propagation feature map, which is input into the emergency state discrimination module to generate a recognition result. When the triggering conditions are met, the door opening command and the window opening command are output to the execution unit.

3. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 2 is characterized in that: The image frame sequence is generated by decoding and separating the onboard video stream. The sensor data includes the instantaneous acceleration value output by the accelerometer, the sound intensity value collected by the microphone, and the door lock status and window control status signals recorded by the onboard control system. The image frame sequence and the sensor data are aligned by timestamp to construct a time-synchronized data input structure.

4. The artificial intelligence-based automobile emergency door and window opening system according to claim 2, characterized in that: The S3 specifically includes: S31, extracting the current image frame, the previous image frame, and the next image frame from the image frame sequence, and inputting them into a feature extraction network composed of a convolution unit, a batch normalization unit, and a channel attention unit, respectively, to generate a feature map of the current image frame, a feature map of the previous image frame, and a feature map of the next image frame; S32, calculate the structural residual map R between the previous image frame and the current image frame at each spatial position (x, y) f (x, y), and the structural residual map R between the next image frame and the current image frame b (x, y), the structural residual graph is calculated using the cosine similarity guidance method: Among them, R(x,y) represents the pixel value of the structure residual map at position (x,y), Represents the feature value of the previous image frame or the next image frame at the channel index c and position (x, y). Represents the feature value of the current image frame at the channel index c and position (x, y), where C represents the number of channels in the image feature map; S33, structural residual graph R f (x,y) and R b (x, y) performs average pooling operation to generate forward response strength and backward response strength respectively, and calculates forward propagation weight and backward propagation weight according to the response strength; S34, applying corresponding propagation weights to the feature map of the previous image frame and the feature map of the next image frame respectively and performing weighted fusion to generate a time fusion feature map, and splicing it with the feature map of the current image frame in the channel dimension to form a combined feature map; S35, structural residual graph R f (x,y) and R b (x, y) is summed and normalized position by position to generate a spatial attention map, and spatial weighting is applied to the combined feature map to generate a residual control feature map; S36. Input the residual control feature map into the channel attention module to generate a time-sensitive spatial feature map.

5. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 2 is characterized in that: The S4 specifically includes: S41. Construct an event graph based on the time-synchronized sensor data. The event graph consists of a set of event nodes and a set of edges. The event nodes represent the states of acceleration, sound intensity, and door and window status signals at a certain point in time, and the edge sets represent the sequential relationship between states at adjacent time points. S42, encoding the spatial feature map of the current frame in the image frame sequence into a set of feature vectors, where each spatial position corresponds to a feature vector; S43. Based on the structural similarity between the spatial feature vector and the event node embedding, a graph mapping tensor M(i, j) is constructed: Among them, M(i,j) represents the matching weight between the i-th spatial feature position and the j-th event node, s i represents the eigenvector of the i-th spatial position, e j represents the embedding representation of the j-th event node, γ represents the scaling factor, and cos(·,·) represents the cosine similarity; S44. Input the graph mapping tensor M(i, j) into the gate structure, and combine it with the current frame spatial feature map to generate a gate tensor for describing the response distribution of the event graph in the spatial structure.

6. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 5, characterized in that: The S44 specifically includes: S441, the gate control structure is a module composed of a graph response generation unit, a feature fusion unit and an adjustment calculation unit, which is used to receive the graph mapping tensor and the spatial feature map, and establish a response relationship between the event graph node and the graph mapping tensor at each image space position; S442, the graph response generation unit calculates the graph response result of each spatial position based on the graph mapping tensor and the embedded representation of the event graph node; S443, the feature fusion unit concatenates the image response result with the spatial feature image vector of the corresponding spatial position to form a joint feature representation; S444, the adjustment calculation unit performs linear transformation and activation function calculation on the joint feature representation to generate a gated response value as an adjustment coefficient of the spatial position; S445. Map the gated response value to the channel dimension of the spatial feature map, perform weighted processing on the image feature vector at each spatial position, and output a gated tensor for adjusting the response intensity distribution of the image spatial position in the structured propagation network.

7. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 2, characterized in that: The S5 specifically includes: S51, taking the spatial position and channel dimension features of each pixel in the spatial feature map as input, and dividing the semantic region by feature similarity and spatial connectivity, where the semantic region is a set of two-dimensional sub-regions consisting of continuous pixel positions; S52. Construct a regional topology map in each semantic region, where nodes of the regional topology map represent pixel position indexes, and edge connections represent connections with spatial contact relationships between nodes. S53, extracting the main direction vector inside the semantic region, calculating the angle between the spatial direction vector connected by each edge and the main direction, and generating a direction consistency score matrix; S54: Setting an angle threshold to determine whether the edge connection direction and the main direction meet the consistency condition, and removing edge connections that do not meet the condition; S55. Update the edge connection direction based on the direction consistency judgment result to generate a regional topology map after structural adjustment.

8. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 2 is characterized in that: The S6 specifically includes: S61, receiving the structural residual result, and calculating an anomaly score map based on the residual amplitude between pixels, where each pixel position in the anomaly score map corresponds to an anomaly score value; S62: Set an abnormality score threshold τ, mark pixel positions with a score value greater than τ as abnormal areas, and generate an abnormality mask map; S63. Extract the abnormal region boundary information from the abnormal mask image, generate a boundary confidence map based on the image gradient, boundary closure, and path continuity, and define the boundary confidence value: b i =α1·g i +α2·c i +α3·p i ; Among them, β i is the confidence value of pixel i in the boundary confidence map, g i is the image gradient value, c i is the boundary closure index, p i is the continuity score of the propagation path, α1, α2, and α3 are weighting coefficients, satisfying α1+α2+α3=1; S64. Select the location with the largest score value in the anomaly score map as the propagation starting point, and perform a reverse multi-scale propagation operation in combination with the boundary confidence map; S65. Limit the propagation path at each scale, skip locations where the boundary confidence value is lower than the lower limit, and output an abnormal repair map after propagation adjustment.

9. The artificial intelligence-based emergency door and window opening system for automobiles according to claim 2, characterized in that: The S7 specifically includes: S71. Fusing the time-sensitive spatial feature map generated by the residual-driven bidirectional temporal control module, the gated tensor generated by the event map matching gating module, the regional topology map generated by the structure-preserving regional propagation module, and the anomaly score map and boundary confidence map generated by the confidence-guided anomaly repair module, aligning them according to the image frame index and splicing them in the channel dimension to generate a fused propagation feature map. S72, performing feature extraction and vector mapping operations on the fused propagation feature map to generate a state response vector; S73. Set a trigger threshold, perform weighted calculation on each component in the state response vector to obtain a score value, and compare the score value with the trigger threshold to determine whether the instruction generation condition is met; S74. When the score value satisfies the instruction generation condition, a door opening instruction and a window opening instruction are generated and sent to the execution units of the door control system and the window control system respectively.

Images