A method for intelligent identification of fire traces in electric vehicles

Abstract

This invention discloses an intelligent identification method for fire traces in electric vehicles. This method intelligently identifies fire traces on three parts of the electric vehicle: its exterior, battery pack, and interior, and performs comprehensive analysis to determine the final location of the fire source. Applying this intelligent fire trace identification system to electric vehicle fire scene investigations improves the speed and accuracy of analyzing the causes of accidents and faulty battery packs, realizing intelligent fire investigation technology. This has significant research and application value for improving the efficiency of on-site investigations. Furthermore, it can be extended to other fields for comprehensive analysis of the ignition point of electric vehicle fires.

Description

Technical Field

[0001] This invention relates to electric vehicle fire detection technology, specifically to an intelligent method for identifying traces of electric vehicle fires. Background Technology

[0002] Currently, electric vehicle fires are increasingly being brought to the forefront of public attention. Fire departments, car manufacturers, and insurance companies are all still in the exploratory stage of investigating electric vehicle fires. The identification of fire traces in electric vehicles is essentially nonexistent; existing identification technologies rely solely on practical experience, requiring a large number of fire investigators to survey the scene and conduct analysis, which consumes enormous human and material resources. Furthermore, the low efficiency of traditional manual trace identification means that many cases cannot fully learn from the experiences of others. Therefore, utilizing computer and deep learning technologies can effectively improve work efficiency.

[0003] A review of domestic and international literature reveals that there are currently no mature and reliable technologies for intelligent identification of fire traces in electric vehicles. Therefore, existing technologies cannot be directly used to assist in processing relevant cases. Consequently, there is an urgent need to summarize and develop identification methods for fire traces in electric vehicles. With the development of artificial intelligence technology, image recognition technology based on convolutional neural networks can be applied to the identification of post-fire image traces in electric vehicles, determining the location of the fire and assisting fire investigators in locating the point of origin. Summary of the Invention

[0004] In view of the current state of technology, this invention proposes an intelligent method for identifying fire traces in electric vehicles. This method employs convolutional neural network (CNN) artificial intelligence technology, which is mature and widely applied across various industries. Due to the specific nature of the firefighting field, the application of artificial intelligence in firefighting is currently limited, particularly in the identification and analysis of fire traces. Furthermore, a large part of artificial intelligence applications are based on image processing, and electric vehicle fire investigation sites generate a large number of trace images, which aligns well with deep learning's use of image data for training.

[0005] The technical solution adopted in this invention is: an intelligent identification method for fire traces in electric vehicles. The method intelligently identifies fire traces on three parts of the electric vehicle: its exterior, battery pack, and interior, and performs comprehensive analysis and judgment to determine the final location of the fire source. The specific steps are as follows:

[0006] I. Exterior of Electric Vehicles

[0007] A1. Establishment of the Map Traces Dataset

[0008] The severity of fire damage in the collected images of electric vehicles is categorized into multiple classes. Each class is labeled at the pixel level, and the labeled images and their corresponding labels are used as the original image trace dataset. The original image trace dataset of fire-damaged electric vehicles collected by the camera is then enhanced and used as the image trace dataset input when building the network model.

[0009] A2. Construction of Network Model

[0010] The network model is a semantic segmentation convolutional neural network combining ResNet-50 with dilated convolutions and UpperNet. The image trace dataset is input into the network model, and the parameters are iteratively updated to complete the construction of the network model. When performing trace analysis on the image, the input of the network model is a single image of the electric vehicle exterior to be analyzed, and the output is a label map of the severity of the fire marks corresponding to the electric vehicle exterior image data.

[0011] A3. Label Image Post-processing

[0012] The label image region of the fire mark location severity output by the network model is decomposed into binary images under the corresponding category. Then, under the corresponding category, multiple label image regions that are close to each other but not completely connected in the binary image are filled into a whole label image region. The weight value is generated by the number of pixels in the whole label image region and the quantized severity.

[0013] A4. Weight Value Calculation

[0014] Image data collected from the front, rear, left, and right sides of the electric vehicle's exterior are input into the constructed network model. A three-row, three-column weight matrix representing the overall severity of fire marks on the electric vehicle is generated based on the weight values ​​produced by post-processing the labeled images. The location of the fire source and suspected fire source is then determined according to the weight values ​​in the weight matrix.

[0015] II. Electric Vehicle Battery Pack

[0016] B1. Determine the relative position of the battery pack on the chassis of the electric vehicle, and determine whether the combustion status of the battery pack matches the combustion status of the entire vehicle; if the battery pack is in the center of the electric vehicle, then the fire distribution of the battery pack is equivalent to the fire distribution of the entire vehicle; otherwise, the battery pack only represents the fire distribution of its local location within the vehicle.

[0017] B2. Obtain the relative position of the battery pack on the electric vehicle chassis from the acquired electric vehicle chassis image. If the vertical coordinate offset of the geometric center of the battery pack outline marked in the electric vehicle chassis image is less than a threshold, the battery pack is considered to be located in the center of the electric vehicle chassis. Otherwise, the relative position is determined according to the sign of the threshold. If the threshold is positive, the battery pack is located in front of the electric vehicle chassis. If it is negative, the battery pack is located behind the electric vehicle chassis.

[0018] B3. Select the four corners of the battery pack to obtain a quadrilateral representing the outline of the battery pack, and perform perspective transformation on the quadrilateral to obtain a rectangular outline of the battery pack surface image.

[0019] B4. Based on the characteristics of the battery pack surface image, the battery pack shell is divided into metal and plastic materials. Thresholds are set according to the brightness and saturation after burning of different materials. The battery pack surface image is then binarized using the thresholds to generate corresponding brightness and saturation masks. The Static Saliency Fine Grained algorithm is used to binarize the output saliency image to form a saliency mask.

[0020] B5. Perform morphological dilation on the brightness mask, saturation mask, and saliency mask, and superimpose the results of the three masks after the morphological operation to extract the intersection region of the three masks, the intersection region of the two masks, and the region represented by only one mask.

[0021] B6. Divide the battery pack image into three rows and three columns of regions with equal areas. Generate weight values ​​by using the number of overlapping masks in each region and the number of pixels in the overlapping parts. These weight values ​​are then mapped to a three-row, three-column weight matrix. The location of the fire source is determined according to the magnitude of the weight values.

[0022] III. Interior of Electric Vehicles

[0023] C1. The input images of the interior of the electric vehicle are the cabin, front row, and rear row. The input images are labeled as four levels: severe, moderate, minor, and intact. The InceptionV3 deep learning-based classification network is used to construct the network model.

[0024] C2. The completed network model is used to determine the severity of fire marks in the input image, and weight values ​​for the corresponding fire mark positions are generated according to the severity of the fire marks, corresponding to a three-row, three-column weight matrix with equal weight values ​​in each row.

[0025] IV. Comprehensive Analysis and Judgment

[0026] D1. Based on the relative position of the battery pack marked in the electric vehicle battery pack section on the electric vehicle chassis, transform the values ​​of each weight in the weight matrix of the electric vehicle battery pack section to form the transformed weight matrix of the electric vehicle battery pack section.

[0027] D2. Add the transformed weight matrix of the battery pack part to the weight matrix of the electric vehicle exterior part and the weight matrix of the vehicle interior to obtain the preliminary fire source location analysis conclusion.

[0028] D3. The preliminary fire source location analysis conclusions and the weight matrix inside the car are comprehensively analyzed and judged to obtain the final fire source location.

[0029] The beneficial effects of this invention are as follows: This invention is an innovative development, which not only has direct positive significance for handling electric vehicle fire cases, but also, through analysis and summarization using computer software, can better promote the construction of smart fire protection.

[0030] This invention applies the developed intelligent electric vehicle trace recognition system to on-site investigations of electric vehicle fires, improving the speed and reliability of analyzing the causes of accidents and faulty battery packs. It realizes intelligent fire exploration technology, which has significant research and application value for improving the speed of fire emergency management and guiding the safety and fire protection technology of new energy vehicles. Furthermore, it can be extended to other fire protection fields, allowing for better preventative measures through comprehensive analysis of past cases.

[0031] This invention provides image trace recognition software to assist in electric vehicle fire investigations. By uploading photos of burn marks on the vehicle and battery pack, it calculates the location of the fire in real time. Using images of the burned electric vehicle as data for processing allows for convenient image collection and storage. Processing and analyzing images through neural networks reduces manpower, improves efficiency, and enhances the accuracy of analysis. The neural network model allows for better optimization and improvement; by continuously expanding the amount of new image data, the model can be improved, facilitating upgrades and optimization. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the comprehensive analysis and judgment process of this invention. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0034] This invention intelligently identifies fire traces in three parts of an electric vehicle: its exterior, battery pack, and interior. A comprehensive analysis is then performed to determine the final location of the fire source. The specific steps in this embodiment are as follows:

[0035] I. Exterior of Electric Vehicles

[0036] A1. Establishment of the Map Traces Dataset

[0037] The severity of fire damage in the collected images of electric vehicles was categorized into multiple classes. Each class was labeled at the pixel level. The labeled images and their corresponding labels were used as the original image trace dataset. The original image trace dataset of fire-damaged electric vehicles collected by the camera was then augmented and used as the image trace dataset input when building the network model.

[0038] In step A1, the severity of fire damage in the collected electric vehicle exterior images is categorized into four types based on different visual representations: intact, slightly damaged, moderately damaged, and severely damaged. Different colors are used to represent these categories, and pixel-level annotations are performed. That is, different colored pixel areas represent fire damage traces of corresponding severity. The annotated images and their corresponding annotations are used as the original dataset.

[0039] Next, the labeled images are processed. The enhancement methods for the original dataset of burned electric vehicle images captured by the camera include horizontal flipping, small-angle random rotation, random adjustment of brightness, random adjustment of contrast, and various combinations of enhancement methods. In other words, the original dataset of burned electric vehicle images captured by the camera is enhanced and used as the input dataset when building the network model.

[0040] A2. Construction of Network Model

[0041] The network model is a semantic segmentation convolutional neural network combining ResNet-50 with dilated convolutions and UpperNet. The image trace dataset is input into the network model, and the parameters are updated iteratively to complete the construction of the network model. When performing trace analysis on images, the input of the network model is a single image of the electric vehicle exterior to be analyzed, and the output is a label image of the severity of the fire marks corresponding to the electric vehicle exterior image data.

[0042] This step selects a semantic segmentation convolutional neural network model that combines ResNet-50 with dilated convolutions and UpperNet. ResNet-50 is a feature extraction network capable of extracting various information from the input image, while the addition of dilated convolutions mitigates the spatial information loss caused by oversampling in the ResNet structure. The UpperNet structure can fuse feature maps at multiple spatial scales, thus the output image of this network model can fully utilize spatial information. The augmented dataset is input into this network model, and parameters are iteratively updated to complete the construction of the network model.

[0043] A3. Label Image Post-processing

[0044] The label image region representing the severity of fire marks output by the network model is decomposed into binary images of the corresponding categories. Then, within each category, multiple label image regions that are close to each other but not completely connected are filled into a single label image region. Weight values ​​are generated using the number of pixels in the single label image region and the quantized severity.

[0045] In step A3, the method for calculating the weight value using the number of pixels in the overall label image region and the quantized severity value is as follows:

[0046] The label image region representing the severity of fire marks output by the network model is divided into three equal parts: left, middle, and right, denoted by a, b, and c respectively. Assuming the number and corresponding weight value for the severity category of "intact" are both 0, then the numbers and corresponding weight values ​​for the severity categories of "minor fire," "moderate fire," and "severe fire" are 1, 2, and 3 respectively. Then, the number of pixels in each of the three equal parts labeled with numbers 0, 1, 2, and 3 are counted, denoted as N0, N1, N2, and N3 respectively. The weight value of each part in the label image output by the network model is calculated using the following formula:

[0047]

[0048] In the formula, i takes the values ​​0, 1, 2, 3, and N i This represents the number of pixels in different numbered regions within one of the equal parts. The network model outputs three parts, a, b, and c, of a single image, whose weights can be represented as W. a W b W c .

[0049] After constructing the network model and obtaining the labeled images representing different severity levels from the network model output, the regions representing "each severity level" in the labeled images are decomposed into one-hot encodings for the corresponding category (i.e., for each severity level, 1 represents the presence of a region with that severity level, and 0 represents the absence of that severity level). Then, a morphological closing operation is performed on the labeled images to fill multiple small regions that are relatively close but not completely connected into a whole. Next, the regions under different severity categories are sorted in order of severity as the primary factor and region size as the secondary factor, resulting in the region at the top of the sort as the fire source location, and the subsequent regions as suspected fire source locations.

[0050] A4. Weight Value Calculation

[0051] Image data collected from the front, rear, left, and right sides of the electric vehicle's exterior are input into the constructed network model. A three-row, three-column weight matrix representing the overall severity of fire marks on the electric vehicle is generated based on the weight values ​​produced by post-processing the labeled images. The location of the fire source and suspected fire source is then determined according to the weight values ​​in the weight matrix.

[0052] In step A4, the three-row, three-column weight matrix representing the overall severity of fire marks on the electric vehicle is generated as follows:

[0053] The front, rear, left, and right images of the electric vehicle's exterior are labeled F, B, L, and R, respectively. The output images in these four directions are divided into three equal parts: a, b, and c. Considering overlap, the third part (labeled F, c) taken from the front of the vehicle and the first part (labeled L, a) taken from the left side of the vehicle together represent the actual front left position of the vehicle. The labels F, B, L, and R in each direction are paired with the three parts a, b, and c to represent nine regions of the electric vehicle's exterior: front left, front center, front right, left center, center, right center, rear left, rear center, and rear right. These nine regions are then mapped sequentially from top to bottom and left to right to a 3x3 weight matrix. The weight values ​​in this matrix represent the severity of the quantified fire marks in each region. The weight values ​​are named W1 to W9 and calculated using the following formulas:

[0054]

[0055] W2 = W F,c

[0056]

[0057] W4 = W L,b

[0058] W6 = W R,b

[0059]

[0060] W8 = W B,b

[0061]

[0062]

[0063] In the formula, i ranges from 1 to 9, representing the number of the 9 weight values ​​in the weight matrix.

[0064] Since the image cannot accurately represent the exact center of the vehicle, the weight value W5 (the element numbered 5 at the very center) is set to the arithmetic mean of its eight adjacent elements. Next, the weight values ​​W in the weight matrix are... i Transforming the values ​​to the range of 0 to 3, the transformed weights are represented as W. i Let W' be an example of a ''. max If the value is the maximum among W1 to W9, then the transformed weight value W i The specific calculation formula for ′ is:

[0065]

[0066] This method intelligently identifies fire traces on the exterior of electric vehicles. It uses a semantic segmentation network combining ResNet50 and UpperNet structures with dilated convolutions to analyze and process the input vehicle exterior image into label maps representing different degrees of severity. The post-processing algorithm analyzes the label maps from multiple angles to finally obtain the possible fire source location.

[0067] II. Electric Vehicle Battery Pack

[0068] The algorithm in this section utilizes image data feature extraction and analysis methods primarily based on brightness, saturation, and saliency features to analyze the target battery pack and determine the most probable fire source location. The specific implementation steps of this embodiment are as follows:

[0069] B1. Determine the relative position of the battery pack on the chassis of the electric vehicle and judge whether the combustion status of the battery pack matches the combustion status of the whole vehicle. If the battery pack is in the center of the electric vehicle, the fire distribution of the battery pack is the same as the fire distribution of the whole vehicle. Otherwise, the battery pack only represents the fire distribution of its local location within the whole vehicle.

[0070] B2. Obtain the relative position of the battery pack on the electric vehicle chassis from the acquired images of the electric vehicle chassis. If the offset of the geometric center of the battery pack outline marked in the electric vehicle chassis image in the vertical direction is less than the threshold, the battery pack is considered to be located in the center of the electric vehicle chassis. Otherwise, the relative position is determined according to the sign of the threshold. If the threshold is positive, the battery pack is located in front of the electric vehicle chassis. If it is negative, the battery pack is located behind the electric vehicle chassis.

[0071] To obtain the relative position of the battery pack on the electric vehicle chassis, it is necessary to prepare images of the chassis taken from both the front and rear, at a fixed angle, height, and distance from the electric vehicle. The battery pack positions in the two images are marked in the software, and the coordinate relationship between the two images is analyzed to obtain the relative position of the battery pack on the electric vehicle chassis. If the vertical coordinate offset of the geometric center of the marked battery pack outline in the front and rear images is less than a threshold, the battery pack is considered to be located in the center of the electric vehicle chassis. Otherwise, the relative position is determined by the sign of the threshold: a positive threshold indicates the battery pack is located at the front of the electric vehicle chassis, and a negative threshold indicates it is located at the rear of the chassis.

[0072] B3. Select the four corners of the battery pack to obtain a quadrilateral representing the outline of the battery pack, and perform perspective transformation on the quadrilateral to obtain a rectangular outline of the battery pack surface image.

[0073] This step requires an image of the burned battery pack as input. The four corners of the battery pack are selected to obtain a quadrilateral that approximates the pack's outline. This quadrilateral is then subjected to perspective transformation to obtain a surface image of the battery pack with a more regular rectangular outline.

[0074] B4. Based on the characteristics of the battery pack surface image, the battery pack shell is divided into metal and plastic materials. Thresholds are set according to the brightness and saturation of different materials after combustion. The battery pack surface image is then binarized using the thresholds to generate corresponding brightness and saturation masks. The Static Saliency Fine Grained algorithm is used to binarize the output saliency image to form a saliency mask.

[0075] In step B4, the image feature criteria for classifying the battery pack casing into metal and plastic materials are as follows: if the average brightness of the electric vehicle battery pack image is higher than a threshold, then the battery pack material is metal; otherwise, it is plastic.

[0076] B5. Perform morphological dilation on the brightness mask, saturation mask, and saliency mask, and then superimpose the results of the three masks after the morphological operation to extract the intersection region of the three masks, the intersection region of the two masks, and the region represented by only one mask.

[0077] This step performs morphological dilation on the three masks, with the mask output by the saliency algorithm as the main one and a larger dilation kernel size set; the brightness mask and saturation mask are used as auxiliary ones and a smaller morphological kernel size is set. The results of the three masks after the morphological operation are superimposed to extract the intersection region of the three masks, the intersection region of the two masks, and the region represented by only one mask.

[0078] B6. Divide the battery pack image into three rows and three columns of regions with equal areas. Generate weight values ​​by using the number of overlapping masks in each region and the number of pixels in the overlapping parts. These weight values ​​are then mapped to a three-row, three-column weight matrix. The location of the fire source is determined according to the magnitude of the weight values.

[0079] In step B6, the generation of the three-row, three-column weight matrix and the calculation of each weight value are as follows:

[0080] The battery pack image is divided into three rows and three columns, corresponding to nine weight values. The weight values ​​in the three-row, three-column weight matrix are named X1 to X9 in order from left to right and top to bottom. The number of pixels in each region that does not overlap with a mask, overlaps with one mask, overlaps with two masks, and overlaps with three masks are L0, L1, L2, and L3, respectively. The weight value of each region is calculated based on these pixel counts, using the following formula:

[0081]

[0082] In the formula, j represents the number of overlapping masks, with values ​​of 1, 2, and 3.

[0083] Next, first obtain the maximum value X of the weight. max Then, the weight values ​​X of each of the nine regions are... i Transform to the range of 0-3 to obtain the region mask weight value X. i The calculation formula is:

[0084] X i ′=3×X i / X max

[0085] In the formula, i is the number, which takes values ​​from 1 to 9.

[0086] III. Interior of Electric Vehicles

[0087] C1. The input images of the interior of the electric vehicle are the cabin, front row, and rear row. The input images are labeled as four levels: severe, moderate, minor, and intact. The InceptionV3 deep learning-based classification network is used to construct the network model.

[0088] The input images for this step are interior images of an electric vehicle, including the cabin, front seats, and rear seats. These images are categorized into four levels of severity: severe, moderate, minor, and intact. The InceptionV3 deep learning-based classification network is used for training, and the trained model is then used to determine the severity of burn damage in the input images. The severity levels of severe, moderate, minor, and intact are represented by the numbers 3, 2, 1, and 0, respectively.

[0089] C2. Use the constructed network model to determine the severity of fire marks in the input image, and generate weight values ​​for the corresponding fire mark positions according to the severity of the fire marks, which are then mapped to a three-row, three-column weight matrix with equal weight values ​​in each row.

[0090] The cabin, front row, and rear row can be approximated as reflecting the fire status of the front, middle, and rear of the electric vehicle. Representing the internal image locations and their severity as a 3×3 weight matrix, each row from top to bottom represents: front, middle, and rear. Each row contains the same three elements, using the severity number of the location represented by that row, i.e., 3, 2, 1, and 0. This weight matrix can then be combined with matrices representing the exterior of the electric vehicle and the battery pack for comprehensive analysis.

[0091] In step C2, the three-row, three-column weight matrix is ​​calculated as follows:

[0092] The severity of fire marks in the input image is categorized into four levels: severe, moderate, slight, and intact, represented by the numbers 3, 2, 1, and 0, respectively. The engine compartment, front row, and rear row positions inside an electric vehicle correspond to the front, middle, and rear of the vehicle. In a three-row, three-column weight matrix, each row has the same three weight values:

[0093] Y1 = Y2 = Y3

[0094] Y4 = Y5 = Y6

[0095] Y7 = Y8 = Y9.

[0096] IV. Comprehensive Analysis and Judgment

[0097] Reference Figure 1 After completing the analysis of the first three parts, a comprehensive analysis and judgment will be made on the output results of these parts, which mainly consists of the following steps:

[0098] D1. Based on the relative position of the battery pack marked in the electric vehicle battery pack section on the electric vehicle chassis, transform the values ​​of each weight in the weight matrix of the electric vehicle battery pack section to form the transformed weight matrix of the electric vehicle battery pack section.

[0099] In step D1, the rules for each weight value in the weight matrix of the transformed battery pack are as follows:

[0100] D1-1. If the battery pack is located in the middle of the electric vehicle chassis, its position can reflect the fire situation of the entire electric vehicle body. Therefore, the weight matrix of the battery pack is not transformed, and the region mask weight value X′ is used directly. i express.

[0101] D1-2. If the battery pack is located at the rear of the electric vehicle chassis, the weight matrix of the battery pack can only represent the fire state of the entire rear of the electric vehicle body. The weight values ​​in the weight matrix of the battery pack are transformed according to the following rules:

[0102] X″4=X′4+0.5×X′1

[0103] X″5=X′5+0.5×X′2

[0104] X″6=X′6+0.5×X′3

[0105] X″7=X′7+0.5×X′4

[0106] X″8=X′8+0.5×X′5

[0107] X″9=X′9+0.5×X′6

[0108] In the formula, X″ i This represents the transformed battery pack weight value; X′ i This represents the battery pack weight value before the transformation.

[0109] D1-3. If the battery pack is located at the front of the electric vehicle chassis, the weight matrix of the battery pack can only represent the fire state of the entire front of the electric vehicle body. The weight values ​​in the weight matrix of the battery pack are transformed according to the following rules:

[0110] W″1=W′1+0.5×W′4

[0111] W″2=W′2+0.5×W′5

[0112] W″3=W′3+0.5×W′6

[0113] W″4=W′4+0.5×W′7

[0114] W″5=W′5+0.5×W′8

[0115] W″6=W′6+0.5×W′9

[0116] In the formula, W″ i W′ represents the weight value of the battery pack after transformation. i This represents the weight value of the battery pack before the transformation.

[0117] D2. Add the transformed weight matrix of the battery pack part to the weight matrix of the electric vehicle exterior part and the weight matrix of the vehicle interior to obtain the preliminary fire source location analysis conclusion.

[0118] The weighted matrix of the battery pack section after transformation is added element-wise with the weighted matrices of the exterior and interior of the car to obtain a 3×3 matrix where each element has a value between 0 and 10.5. A suspected fire source threshold of T1 and a confirmed fire source threshold of T2 are set. The actual location of the car represented by the element in the matrix that satisfies both threshold conditions is found, thus obtaining the suspected fire source location and the confirmed fire source location. This conclusion can represent nine locations of the car: front left, front center, front right, left center, center, right center, left rear, rear center, and right rear.

[0119] D3. The preliminary fire source location analysis conclusions are combined with the weight matrix inside the car to determine the final fire source location.

[0120] This step can be divided into the following two cases:

[0121] (1) If the fire source is located in the center of the electric vehicle, the value representing the severity of the fire inside the front row of the electric vehicle is multiplied by the scaling factor γ and compared with the value representing the severity of the fire at the center of the battery pack. If the former value is greater than the latter value, it can be determined that the fire is inside the electric vehicle; otherwise, it is at the center of the bottom of the electric vehicle.

[0122] (2) If the fire source is not in the center of the electric vehicle, it is necessary to trace it back to the corresponding location in the image outside the electric vehicle. Recalculate the weight matrix of the electric vehicle's appearance on one-third of the vehicle body representing the corresponding location, then divide it into three equal parts from top to bottom, calculate the weight values, and determine if the fire location is above, in the middle, or below the electric vehicle. Since the fire source of the electric vehicle is usually at the battery pack location, the weight value at the bottom of the weight matrix of the electric vehicle's appearance needs to be added to the weight value of the corresponding location of the battery pack.

[0123] For example, if it is determined that the fire source is at the right rear of the electric vehicle, an external image showing the right rear of the electric vehicle should be found, and the weight values ​​of the three equal parts from top to bottom should be calculated. Then, the weight value of the corresponding position of the battery pack should be added to the bottom of the right rear external image. By comparing the weight values ​​of the three parts, the conclusion that the fire source is located at the bottom of the right rear of the electric vehicle can be drawn.

[0124] In step D3, the comprehensive analysis and judgment method is divided into the following two cases:

[0125] D3-1. If the preliminary fire source location analysis conclusion obtained in step D2 indicates that the fire source is located in the center of the electric vehicle, then the weight value representing the severity of the front row position inside the electric vehicle is multiplied by a scaling factor, and then compared with the value representing the severity of the center position of the electric vehicle's battery pack. If the scaled value of the front row position inside the electric vehicle is greater than the value of the center position of the battery pack, then the fire location is determined to be inside the vehicle body; otherwise, it is at the center of the bottom of the vehicle body.

[0126] D3-2. If the preliminary fire source location analysis conclusion obtained in step D2 indicates that the fire source location is not in the center of the electric vehicle, then trace back to the label image on the exterior of the electric vehicle that represents the corresponding fire source location. Divide the label image of the corresponding fire source location into three equal parts: upper, middle, and lower. Count the total number of pixels in each equal part from top to bottom in the label image. The part with the largest total number of pixels is the fire source location.

Claims

1. A method for intelligent identification of fire traces in electric vehicles, characterized in that, The method intelligently identifies fire traces on three parts of the electric vehicle: its exterior, battery pack, and interior, and performs comprehensive analysis to determine the final location of the fire source. The specific steps are as follows: I. Exterior of Electric Vehicles A1. Establishment of the Map Traces Dataset The severity of fire damage in the collected images of electric vehicle exteriors is divided into multiple categories. The categories are labeled at the pixel level, and the labeled images and corresponding labels are used as the original image trace dataset. The original image trace dataset of fire-damaged electric vehicles collected by the camera is enhanced and used as the image trace dataset input when building the network model. A2. Construction of Network Model The network model is a semantic segmentation convolutional neural network combining ResNet-50 with dilated convolution and UpperNet; the image trace dataset is input into the network model, and the parameters are iteratively updated to complete the construction of the network model; when performing trace analysis on the image, the input of the network model is a single image of the electric vehicle exterior to be analyzed, and the output is a label image of the severity of the fire mark location corresponding to the electric vehicle exterior image data; A3. Label Image Post-processing The label image region of the fire mark location severity output by the network model is decomposed into binary images under the corresponding category. Then, under the corresponding category, multiple label image regions that are close to each other but not completely connected in the binary image are filled into a whole label image region. The weight value is generated by the number of pixels in the whole label image region and the quantized severity. The method for calculating the weight value using the number of pixels in the overall labeled image region and the quantized severity value is as follows: The label image region of the fire mark location severity output by the network model is divided into three equal parts: left, middle, and right, which are represented by a, b, and c respectively. Assuming that the number representing the severity of the intact category and the corresponding weight value are both 0, then the numbers representing the severity of the minor fire, moderate fire, and severe fire categories and the corresponding weight values ​​are 1, 2, and 3 respectively. Then, the number of pixels in each of the three equal parts of the label image (left, middle, and right) that are represented by different numbered regions (0, 1, 2, and 3) is counted, denoted as N0, N1, N2, and N3 respectively. The weight value of each part in the label image output by the network model is then calculated according to the following formula: In the formula, i takes the values ​​0, 1, 2, 3, and N i This indicates the number of pixels in different numbered regions within one of the equal parts; A4. Weight Value Calculation Image data collected from the front, rear, left and right sides of the electric vehicle's exterior are input into the constructed network model. A three-row, three-column weight matrix representing the overall severity of fire marks on the electric vehicle is generated based on the weight values ​​generated by post-processing of the labeled images. The location of the fire source and suspected fire source is then determined according to the weight values ​​in the weight matrix. The process of generating the three-row, three-column weight matrix representing the overall fire severity of the electric vehicle is as follows: The front, rear, left, and right images of the electric vehicle's exterior are labeled F, B, L, and R, respectively. The output labels for these four directions are divided into three equal parts: a, b, and c. The labels F, B, L, and R are paired with the three parts a, b, and c to represent nine regions of the electric vehicle's exterior: left front, front center, right front, left center, center, right center, left rear, rear center, and right rear. These nine regions are then mapped sequentially from top to bottom and left to right to a 3x3 weight matrix. The weight values ​​in this matrix represent the severity of the fire marks quantified in each region. These weight values ​​are named W1 to W9 and are calculated using the following formulas: W2=W F,c W4=W L,b W6=W R,b W8=W B,b In the formula, i is 1 to 9, representing the number of the 9 weight values ​​in the weight matrix; Next, the weight values ​​W in the weight matrix are... i Transforming the values ​​to the range of 0 to 3, the transformed weights are represented as W. i Let W' be an example of a ''. max If the value is the maximum among W1 to W9, then the transformed weight value W i The specific calculation formula for ′ is: II. Electric Vehicle Battery Pack B1. Determine the relative position of the battery pack on the chassis of the electric vehicle, and determine whether the combustion status of the battery pack matches the combustion status of the whole vehicle; if the battery pack is in the center of the electric vehicle, the fire distribution of the battery pack is equivalent to the fire distribution of the whole vehicle; otherwise, the battery pack only represents the fire distribution of its local location within the whole vehicle. B2. Obtain the relative position of the battery pack on the electric vehicle chassis from the acquired electric vehicle chassis image. If the offset of the geometric center longitudinal coordinate of the battery pack outline marked in the electric vehicle chassis image is less than a threshold, the battery pack is considered to be located in the center of the electric vehicle chassis. Otherwise, the relative position is determined according to the sign of the threshold. If the threshold is positive, the battery pack is located in front of the electric vehicle chassis. If it is negative, the battery pack is located behind the electric vehicle chassis. B3. Select the four corners of the battery pack to obtain a quadrilateral representing the outline of the battery pack, and perform perspective transformation on the quadrilateral to obtain a rectangular outline of the battery pack surface image. B4. Based on the characteristics of the battery pack surface image, the battery pack shell is divided into metal and plastic materials. Thresholds are set according to the brightness and saturation of different materials after combustion. The battery pack surface image is then binarized using the thresholds to generate corresponding brightness and saturation masks. The Static Saliency Fine Grained algorithm is used to binarize the output saliency image to form a saliency mask. B5. Perform morphological dilation on the brightness mask, saturation mask, and saliency mask, and superimpose the results of the three masks after the morphological operation to extract the intersection region of the three masks, the intersection region of the two masks, and the region represented by only one mask. B6. Divide the battery pack image into three rows and three columns of regions with equal areas. Generate weight values ​​by using the number of overlaps of the masks in each region and the number of pixels in the overlapping parts. These weight values ​​are then mapped to a three-row, three-column weight matrix. The location of the fire source is determined according to the magnitude of the weight values. The process of generating the three-row, three-column weight matrix and calculating each weight value is as follows: The battery pack image is divided into three rows and three columns, corresponding to nine weight values. The weight values ​​in the three-row, three-column weight matrix are named X1 to X9 in order from left to right and top to bottom. The number of pixels in each region that does not overlap with a mask, overlaps with one mask, overlaps with two masks, and overlaps with three masks are L0, L1, L2, and L3, respectively. The weight value of each region is calculated based on these pixel counts, using the following formula: In the formula, j represents the number of overlapping masks, and the value is 1, 2, or 3; Next, first obtain the maximum value X of the weight. max Then, the weight values ​​X of each of the nine regions are... i Transform to the range of 0-3 to obtain the region mask weight value X′. i The calculation formula is: X′ i =3×X i / X max In the formula, i is the number, which takes values ​​from 1 to 9; III. Interior of Electric Vehicles C1. The input images of the interior of the electric vehicle are the cabin, front row, and rear row. The input images are labeled as four levels: severe, moderate, slight, and intact. The InceptionV3 deep learning-based classification network is used to construct the network model. C2. The completed network model is used to determine the severity of fire marks in the input image, and weight values ​​for the corresponding fire mark positions are generated according to the severity of the fire marks, corresponding to a three-row, three-column weight matrix with equal weight values ​​in each row. IV. Comprehensive Analysis and Judgment D1. Based on the relative position of the battery pack marked in the electric vehicle battery pack section on the electric vehicle chassis, transform the values ​​of each weight in the weight matrix of the electric vehicle battery pack section to form the transformed weight matrix of the electric vehicle battery pack section. D2. Add the weighted matrix of the battery pack section after transformation to the weighted matrix of the electric vehicle exterior section and the weighted matrix of the vehicle interior section to obtain preliminary conclusions on the fire source location analysis. D3. The preliminary fire source location analysis conclusions and the weight matrix inside the car are comprehensively analyzed and judged to obtain the final fire source location. The comprehensive analysis and judgment method is divided into the following two cases: D3-1. If the preliminary fire source location analysis conclusion obtained in step D2 indicates that the fire source is located in the center of the electric vehicle, then the weight value representing the severity of the front row position inside the electric vehicle is multiplied by a scaling factor, and then compared with the value representing the severity of the center position of the electric vehicle battery pack. If the scaled value of the front row position inside the electric vehicle is greater than the value of the center position of the battery pack, then the fire location is determined to be inside the vehicle body; otherwise, it is at the center of the bottom of the vehicle body. D3-2. If the preliminary fire source location analysis conclusion obtained in step D2 indicates that the fire source location is not in the center of the electric vehicle, then trace back to the label image on the exterior of the electric vehicle that represents the corresponding fire source location. Divide the label image of the corresponding fire source location into three equal parts: upper, middle, and lower. Count the total number of pixels in each equal part from top to bottom in the label image. The part with the largest total number of pixels is the fire source location.

2. The intelligent identification method for fire traces in electric vehicles according to claim 1, characterized in that, In step A1, the severity of the fire damage in the collected images of the electric vehicle exterior is divided into four categories: intact, slightly damaged, moderately damaged, and severely damaged. Different colors are used to represent the label categories, and pixel-level annotations are performed.

3. The intelligent identification method for fire traces in electric vehicles according to claim 1, characterized in that, In step A1, the enhancement processing method for the original burned electric vehicle image dataset acquired by the camera includes horizontal flipping, random rotation at a small angle, random adjustment of brightness, random adjustment of contrast, and various combinations of the enhancement processing methods.

4. The intelligent identification method for fire traces in electric vehicles according to claim 1, characterized in that, In step B4, the image feature criteria for classifying the battery pack casing into metal and plastic materials are as follows: if the average brightness of the electric vehicle battery pack image is higher than a threshold, then the battery pack material is metal; otherwise, it is plastic.

5. The intelligent identification method for fire traces in electric vehicles according to claim 1, characterized in that, In step C2, the three-row, three-column weight matrix is ​​calculated as follows: The four categories for judging the severity of fire marks in the input image—severe, moderate, slight, and intact—are represented by the numbers 3, 2, 1, and 0, respectively. The engine compartment, front row, and rear row positions inside the electric vehicle correspond to the front, middle, and rear of the electric vehicle, respectively. The three weight values ​​in each row of the three-row, three-column weight matrix are identical: Y1 = Y2 = Y3 Y4 = Y5 = Y6 y7 = Y8 = Y9.

6. The intelligent identification method for fire traces in electric vehicles according to claim 1, characterized in that, In step D1, the rules for each weight value in the weight matrix of the transformed battery pack are as follows: D1-1. If the battery pack is located in the middle of the electric vehicle chassis, its position can reflect the fire situation of the entire electric vehicle body. Therefore, the weight matrix of the battery pack is not transformed, and the region mask weight value X′ is used directly. i express; D1-2. If the battery pack is located at the rear of the electric vehicle chassis, the weight matrix of the battery pack can only represent the fire state of the entire rear of the electric vehicle body. The weight values ​​in the weight matrix of the battery pack are transformed according to the following rules: X″4=X′4+0.5×X′1 X″5=X′5+0.5×X′2 X″6=X′6+0.5×X′3 X″7=X′7+0.5×X′4 X″8=X′8+0.5×X′5 X″9=X′9+0.5×X′6 In the formula, X″ i This represents the transformed battery pack weight value; X′ i This represents the battery pack weight value before the transformation; D1-3. If the battery pack is located at the front of the electric vehicle chassis, the weight matrix of the battery pack can only represent the fire state of the entire front of the electric vehicle body. The weight values ​​in the weight matrix of the battery pack are transformed according to the following rules: W″1=W′1+0.5×W′4 W″2=W′2+0.5×W′5 W″3=W′3+0.5×W′6 W″4=W′4+0.5×W′7 W″5=W′5+0.5×W′8 W″6=W′6+0.5×W′9 In the formula, W″ i W′ represents the weight value of the battery pack after transformation. i This represents the weight value of the battery pack before the transformation.

Images