An image processing-based new energy vehicle chassis defect detection system and method

Abstract

This application provides an image processing-based system and method for detecting defects in the chassis of new energy vehicles. The system acquires chassis images of new energy vehicles entering a battery swapping station and initializes a small sample support set. The chassis images are then input into a dual-path feature extraction network. Based on a prototype network, defect prototype features from the small sample support set are extracted to obtain a defect category map. Dual-perception attention is used to suppress background interference in the chassis images and enhance the localization of defect targets, resulting in a localization feature map. The fused feature map and the small sample support set are evaluated using an introspection metric to obtain a defect evaluation value. When the defect evaluation value exceeds a rejection threshold, an unknown defect is determined in the battery pack protection plate of the chassis, triggering a rejection signal. Using the scheme of this application, background interference can be effectively suppressed under small sample conditions, enabling accurate identification and localization of rare defects in the battery pack protection plate of new energy vehicle chassis.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a new energy vehicle chassis defect detection system and method based on image processing. Background Technology

[0002] With the popularization of battery swapping for new energy vehicles, battery swapping stations have placed higher demands on the rapid and accurate detection of vehicle chassis conditions. As a key component protecting the battery pack, the battery pack protective plate in the chassis may cause damage to the battery pack during subsequent use and lead to safety hazards if its surface defects are not detected in time. Therefore, automatic and efficient detection of defects in the battery pack protective plate during the battery swapping process is of great significance for ensuring battery swapping safety and vehicle reliability.

[0003] Existing chassis defect detection methods mostly rely on manual visual inspection or traditional image processing techniques, which are inefficient and prone to missed detections. Although deep learning-based detection methods have been proposed in recent years, these methods typically require a large number of labeled samples for training. However, in the actual operation of battery swapping stations, there are very few samples of rare defects in battery pack protective panels, making it difficult to collect a sufficient amount of training data. This results in insufficient generalization ability and low detection accuracy of conventional deep learning models. At the same time, interference from complex backgrounds in chassis images further increases the difficulty of accurate defect localization and classification. Therefore, how to effectively suppress background interference under small sample conditions and achieve accurate identification and localization of rare defects in battery pack protective panels in new energy vehicle chassis has become a challenge for the industry. Summary of the Invention

[0004] This application provides a new energy vehicle chassis defect detection system and method based on image processing, which can effectively suppress background interference under small sample conditions and achieve accurate identification and location of rare defects in the battery pack protection plate in the new energy vehicle chassis.

[0005] In a first aspect, this application provides a method for detecting defects in the chassis of new energy vehicles based on image processing, including the following steps: Collect chassis images of new energy vehicles when they enter battery swapping stations, and initialize a small sample support set for rare chassis defects of new energy vehicles; The chassis image is input into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the class prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect. The defect category map and the location feature map are fused together, and then the fused feature map and the small sample support set are evaluated by introspection metric to obtain the defect evaluation value of the chassis battery pack protection plate. When the defect assessment value exceeds the rejection threshold, it is determined that the battery pack protection plate of the chassis has an unknown defect and a rejection signal is triggered.

[0006] In some embodiments, acquiring chassis images of new energy vehicles when they enter a battery swapping station and initializing a small sample support set for rare chassis defects of new energy vehicles specifically includes: When a new energy vehicle enters the battery swapping channel of the battery swapping station, it triggers multiple industrial cameras deployed in the pit to take pictures simultaneously. Acquire an image of the area where the battery pack protective plate is located in the chassis of a new energy vehicle, wherein the image at least covers the bottom surface and side edges of the protective plate; The image is subjected to illumination equalization and noise reduction processing to obtain a chassis image; Rare defect samples of labeled battery pack protection panels are selected from historical battery swapping records. Each sample contains a defect image and corresponding defect category label and edge contour annotation, thus forming a small sample support set.

[0007] In some embodiments, extracting defect prototype features of the small sample support for the battery pack protection plate based on a class prototype network in the semantic edge path to obtain a defect category map specifically includes: The defect category set obtained from the small sample support set, each defect category contains at least one labeled defect image of the battery pack protective plate and the corresponding defect edge mask; All defect images are input into the backbone network of semantic edge paths, and support samples are extracted for each defect category; Based on all the defect edge masks, mask average pooling is performed on all the support samples to obtain multiple defect prototype features of the battery pack protection plate. Extract the query feature map of the chassis image, and then determine the semantic distance with each defect prototype feature to obtain the probability response map; The probability response map and the query feature map are dynamically fused using semantic attention to obtain a defect category map of the battery pack protection board.

[0008] In some embodiments, employing dual-sensory attention in the defect detection path to suppress background interference in the chassis image and enhance the localization of the protective plate defect target, the resulting localization feature map of the chassis protective plate defect specifically includes: The chassis image is input into the main path of the defect detection path to extract the original feature map; The foreground and background at each spatial location in the original feature map are distinguished, and the response value of the background region is reduced to obtain a background-suppressed discriminative feature map. Determine the spatial correlation matrix between each pixel in the discriminative feature map to obtain the spatial attention weight map of the enhanced protective plate defect target; The spatial attention weight map and all the original feature maps are weighted to obtain the location feature map of the chassis protection plate defect.

[0009] In some embodiments, fusing the defect category map and the localization feature map specifically includes: The defect category map is compressed using a convolutional layer to align its channel count with the channel count of the localization feature map, and a semantic feature tensor is extracted. The localization feature map and the semantic feature tensor are concatenated along the channel dimension to form a joint feature tensor containing semantic category information and spatial location information; The joint feature tensor is input into a bidirectional feature network, and the interaction of multi-scale features is achieved through bidirectional paths to obtain a fused feature image.

[0010] In some embodiments, the introspection metric evaluation of the fused feature map and the small sample support set to obtain the defect evaluation value of the chassis battery pack protection plate specifically includes: Flatten the fused feature map into a sequence of feature vectors; Determine the semantic distance between the feature vector sequence and all defect prototype features; Based on all semantic distance and introspective similarity metrics, the confidence level of each feature vector belonging to each defect category is determined, and the defect evaluation value of the chassis battery pack protection plate is obtained.

[0011] In some embodiments, when the defect assessment value exceeds the rejection threshold, determining that the chassis battery pack protection plate has an unknown defect and triggering a rejection signal specifically includes: Obtain the preset rejection threshold; The defect assessment value is compared with the rejection threshold. If the defect assessment value is greater than the rejection threshold, it is determined that the battery pack protection board has an unknown defect type and a rejection signal is triggered. If the defect evaluation value is less than or equal to the rejection threshold, the defect type is output according to the category with the highest confidence in the defect category map, and the bounding box position of the defect is output according to the localization feature map.

[0012] Secondly, this application provides an image processing-based new energy vehicle chassis defect detection system, comprising: The acquisition module is used to collect chassis images of new energy vehicles when they enter the battery swapping station and initialize a small sample support set of rare defects in the chassis of new energy vehicles. The processing module is used to input the chassis image into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the class prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect. The processing module is also used to fuse the defect category map and the location feature map, and then perform introspection metric evaluation on the fused feature map and the small sample support set to obtain the defect evaluation value of the chassis battery pack protection plate. The execution module is used to determine that the chassis battery pack protection plate has an unknown defect and trigger a rejection signal when the defect evaluation value exceeds the rejection threshold.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described image processing-based new energy vehicle chassis defect detection method.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing-based method for detecting defects in the chassis of new energy vehicles.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: This application provides a new energy vehicle chassis defect detection system and method based on image processing. First, chassis images of the new energy vehicle entering a battery swapping station are acquired, and a small sample support set for rare chassis defects is initialized. The chassis images are then input into a dual-path feature extraction network, which extracts defect prototype features of the battery pack protection plate from the small sample support set based on a class prototype network in the semantic edge path, obtaining a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis images and enhance the localization of the protection plate defect target, obtaining a localization feature map of the chassis protection plate defect. The defect category map and the localization feature map are fused, and then the fused feature map and the small sample support set are evaluated using an introspection metric to obtain a defect evaluation value for the chassis battery pack protection plate. When the defect evaluation value exceeds a rejection threshold, an unknown defect is determined to exist in the chassis battery pack protection plate, triggering a rejection signal.

[0016] Therefore, in the process of the image processing-based new energy vehicle chassis defect detection method of this application, firstly, the chassis image is input into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the class prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect. Among them, the defect prototype feature is a vector obtained by averaging the feature maps of all support samples under the same defect category according to the corresponding defect edge mask, and then extracting the defect region feature vector. This vector is used to represent the defect of this type in the feature space. The semantic center is used to facilitate distance comparison with the query feature map of the image to be detected, enabling defect category matching under small sample conditions. The localization feature map is obtained by element-wise weighting of the spatial attention weight map and the original defect detection feature map, and is used for subsequent fusion to provide spatial distribution information of defect candidate locations, complementing the category information output by the semantic edge path, and achieving joint optimization of classification and localization. Next, the defect category map and the localization feature map are fused, and then the fused feature map and the small sample support set are evaluated by introspection metric to obtain the defect evaluation value of the chassis battery pack protection plate. When the defect evaluation value exceeds the rejection threshold, it is determined that the chassis battery pack protection plate has an unknown defect and a rejection signal is triggered. The above scheme can effectively suppress background interference under small sample conditions and achieve accurate identification and localization of rare defects in the battery pack protection plate of new energy vehicle chassis. Attached Figure Description

[0017] Figure 1 This is an exemplary flowchart of an image processing-based method for detecting defects in the chassis of new energy vehicles, according to some embodiments of this application. Figure 2 These are structural example diagrams of a new energy vehicle chassis shown in some embodiments of this application; Figure 3 This is a flowchart illustrating the feedback of rejection signals according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of a new energy vehicle chassis defect detection system based on image processing, according to some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing an image processing-based method for detecting defects in the chassis of a new energy vehicle, according to some embodiments of this application. Detailed Implementation

[0018] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0019] refer to Figure 1 The figure is an exemplary flowchart of an image processing-based method for detecting defects in the chassis of a new energy vehicle, according to some embodiments of this application. This image processing-based method for detecting defects in the chassis of a new energy vehicle mainly includes the following steps: In step 101, chassis images of new energy vehicles are acquired when they enter the battery swapping station, and a small sample support set of rare defects in the chassis of new energy vehicles is initialized.

[0020] In some embodiments, acquiring chassis images of new energy vehicles when they enter a battery swapping station and initializing a small sample support set for rare chassis defects of new energy vehicles can be achieved through the following steps: When a new energy vehicle enters the battery swapping channel of the battery swapping station, it triggers multiple industrial cameras deployed in the pit to take pictures simultaneously. Acquire an image of the area where the battery pack protective plate is located in the chassis of a new energy vehicle, wherein the image at least covers the bottom surface and side edges of the protective plate; The image is subjected to illumination equalization and noise reduction processing to obtain a chassis image; Rare defect samples of labeled battery pack protection panels are selected from historical battery swapping records. Each sample contains a defect image and corresponding defect category label and edge contour annotation, thus forming a small sample support set.

[0021] In practice, when a new energy vehicle enters the battery swapping channel of the battery swapping station, triggering multiple industrial cameras installed in the pit to simultaneously take pictures can be achieved in the following way: when the new energy vehicle enters the battery swapping channel of the battery swapping station and reaches the preset trigger position, the battery swapping station control system simultaneously sends trigger signals to multiple industrial cameras installed in the pit; (Reference) Figure 2 As shown in the figure, this figure is a structural schematic diagram of a new energy vehicle chassis in some embodiments of this application. The structural schematic diagram includes a battery pack protective plate, a motor, a suspension system, and transmission components. Each camera captures a bottom image at the same moment it receives a signal, thereby obtaining raw image data from multiple perspectives. Other embodiments may also use other methods to achieve this, which are not limited here.

[0022] In specific implementation, acquiring an image of the area where the battery pack protective plate is located in the chassis of a new energy vehicle, with the image at least covering the bottom surface and side edges of the protective plate, can be achieved in the following way: From multiple original images simultaneously captured by multiple industrial cameras, select one or more images that best display the bottom surface and left and right side edges of the battery pack protective plate; crop the selected image according to the circumscribed rectangle of the protective plate, retaining the protective plate area and removing irrelevant background parts such as the vehicle frame and the ground, and use the cropped image as the input image for subsequent illumination equalization processing; performing illumination equalization processing and noise reduction processing on the image to obtain the chassis image can be achieved in the following way: For the cropped battery pack protective plate area image, firstly, a histogram equalization method is used to adjust the brightness distribution of the entire image, compensating for overly dark or overly bright areas to achieve illumination equalization. Then, a mean filtering algorithm is applied to remove salt-and-pepper noise and random noise from the image while maintaining the clarity of defect edges; the image obtained after processing is the chassis image; other methods can also be used in other embodiments, which are not limited here.

[0023] In specific implementation, rare defect samples of the battery pack protection panel that have been marked are selected from the historical battery swapping records. Each sample contains a defect image, a corresponding defect category label, and an edge contour annotation, thus forming a small sample support set. This can be achieved in the following way: From the historical battery swapping database of the battery swapping station, all records that have been manually reviewed and confirmed as defects in the battery pack protection panel are retrieved. No more than five samples are selected according to the defect category. Each sample contains the original defect image, the corresponding defect category label, and the defect edge contour annotated with a polygon. These samples are organized into a set, which is the small sample support set. The defect category can be, for example, scratches, cracks, dents, and perforations. Other embodiments can also be implemented in other ways, which are not limited here.

[0024] It should be noted that the chassis image in this application is a standardized preprocessed image used as input to the dual-path feature extraction network, including the protective plate area and removing irrelevant background parts such as the frame and ground; the small sample support set is a collection of a small number of labeled samples selected from historical battery swapping records, containing various rare defects of the battery pack protective plate, used to extract the prototype feature vectors of various defects in the semantic edge path through a prototype-like network, which facilitates the accurate classification and localization of rare defects in newly acquired chassis images with only a small number of labeled samples, and solves the technical problem that it is difficult to collect a large number of rare defect samples in the battery swapping station scenario and that conventional deep learning models cannot be effectively trained.

[0025] In step 102, the chassis image is input into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect.

[0026] It should be noted that the dual-path feature extraction network in this application consists of a semantic edge path and a defect detection path that share the same pre-trained residual network (such as ResNet50) backbone. The semantic edge path inputs defect images from a small sample support set into the backbone network to extract support sample feature maps, and performs mask average pooling in combination with the defect edge mask to calculate the prototype vector of each defect category. Then, the chassis image is input into the same backbone network to obtain the query feature map, and the Euclidean distance between each pixel of the query feature map and the prototype vector is calculated and normalized to generate a probability response map. Then, the probability response map and the multi-scale feature map of the backbone network are weighted and fused through the dynamic semantic attention module to output the defect category map. The defect detection path inputs the chassis image into the same backbone network to obtain the original feature map for defect detection. It then passes through the background attenuation attention module (two 1×1 convolutional layers to generate a foreground confidence map and weight it) and the spatial awareness attention module (the module that calculates the spatial correlation matrix between pixels to generate an attention weight map and weights it), and outputs the localization feature map. The two paths output category semantics and spatial location information respectively, which are then fused. The entire network is trained using a meta-learning strategy, with no more than five supporting samples for each defect category to simulate a small sample scenario of rare defects.

[0027] It should be noted that the semantic edge path in this application consists of a prototype network and a dynamic semantic attention module. The prototype network extracts prototype vectors for each type of defect from small samples, which are then matched with the query feature map of the chassis image to solve the problem of insufficient rare defect samples. The dynamic semantic attention module fuses the matched probability response map with the multi-scale feature map of the backbone network to highlight the semantic information of the defect area and output a pixel-level defect category map. The output of the semantic edge path directly corresponds to the defect type discrimination required by the battery swapping station. The defect detection path consists of a background attenuation attention module and a spatial awareness attention module connected in series. The background attenuation attention module suppresses common background interferences in the chassis image, such as ground, oil stains, and shadows, which are common in battery swapping stations. The spatial awareness attention module calculates the correlation between pixels in the feature map, enhances the localization response of the defect target, and outputs a localization feature map. The output of the defect detection path is used to determine the specific location of the defect on the battery pack protection plate. The two paths share the same backbone network and output the category semantics and spatial location respectively. Subsequently, the defect classification and localization are achieved through fusion.

[0028] In some embodiments, the defect prototype features of the small sample supporting the concentrated battery pack protection plate are extracted based on a class prototype network in the semantic edge path to obtain a defect category map, which can be achieved by the following steps: The defect category set obtained from the small sample support set, each defect category contains at least one labeled defect image of the battery pack protective plate and the corresponding defect edge mask; All defect images are input into the backbone network of semantic edge paths, and supporting samples are extracted for each defect category; Based on all the defect edge masks, mask average pooling is performed on all the support samples to obtain multiple defect prototype features of the battery pack protection plate. Extract the query feature map of the chassis image, and then determine the semantic distance with each defect prototype feature to obtain the probability response map; The probability response map and the query feature map are dynamically fused using semantic attention to obtain a defect category map of the battery pack protection board.

[0029] In specific implementation, the defect category set obtained from the small sample support set, where each defect category contains at least one labeled defect image of the battery pack protective plate and its corresponding defect edge mask, can be implemented in the following way: Traverse all defect samples in the small sample support set, extract the defect category label for each sample (e.g., scratches, cracks, dents, perforations, etc.), remove duplicates from all category labels to form a category list; simultaneously record all sample images contained in each category and the corresponding defect edge mask for each sample. The defect edge mask is a binary image of the same size as the sample image, where a pixel value of 1 indicates that the position of the defect is... Pixels belong to defective regions, and positions with a pixel value of 0 represent non-defective regions. Inputting all defective images into the backbone network of the semantic edge path and extracting support samples for each defect category can be achieved as follows: input each defective image from the small sample support set into the backbone network of the semantic edge path, and calculate the feature map output by the last layer of the backbone network for each defective image through forward propagation. For all defective images under the same defect category, all their corresponding feature maps are aggregated to form the support samples for that defect category. Other implementation methods can also be used in other embodiments, which are not limited here.

[0030] In specific implementation, the process of performing mask average pooling on all support samples based on all defect edge masks to obtain multiple defect prototype features of the battery pack protection plate can be achieved in the following way: For each defect category, extract the support samples of that defect category and the defect edge masks corresponding to each feature map of each support sample; for each support sample's feature map, locate the coordinates on all feature maps of the support sample based on the positions of the pixel values ​​of 1 in the corresponding defect edge mask, and extract the feature vectors at these coordinates; aggregate the feature vectors extracted from the feature maps of all support samples and calculate the average value according to the channel dimension to obtain a defect prototype vector corresponding to that defect category; each channel value of this defect prototype vector represents the center position of that type of defect in the feature space; traverse all defect categories to obtain multiple defect prototype vectors; other implementation methods can also be used in other embodiments, which are not limited here.

[0031] In specific implementation, the query feature map of the chassis image is extracted, and then the semantic distance with each defect prototype feature is determined to obtain the probability response map. This can be achieved in the following way: the preprocessed chassis image is input into the backbone network of the same semantic edge path mentioned above to extract the query feature map of the chassis image; for each pixel position in the query feature map, the Euclidean distance between its feature vector and each defect prototype feature is calculated, where the smaller the Euclidean distance, the greater the probability that the pixel belongs to the defect category; the Euclidean distances of all defect categories are normalized after negative exponential transformation to obtain the probability that each pixel belongs to each defect category; these probabilities are organized into a multi-channel probability response map according to category channels; the probability response map and the... The defect category map of the battery pack protective panel can be obtained by performing dynamic semantic attention fusion on the query feature map, as follows: all probabilities in the probability response map are used as attention weights and multiplied with the query features channel-wise and pixel-wise. Features of regions with high weights, i.e., regions considered to have a high probability of defects, are enhanced, while features of regions with low weights, i.e., background or normal regions, are suppressed. The weighted query features are then passed through a convolutional layer and an upsampling layer to restore the original chassis image size and output a single-channel category label map, i.e., the defect category map of the battery pack protective panel. The value of each pixel directly represents the defect category number to which the pixel belongs. Other embodiments may also use other methods, which are not limited here.

[0032] It should be noted that the defect category set in this application is an enumerated list of all different defect category names, such as scratches, cracks, dents, perforations, and their corresponding samples, extracted from the small sample support set; the support samples are the feature map set output from the last layer of the semantic edge path backbone network after each defect image in the small sample support set is input into the network; the defect prototype feature is the vector obtained by averaging the feature maps of all support samples under the same defect category, after extracting the defect region feature vector according to the corresponding defect edge mask, and then averaging the vector by channel. This vector represents the semantic center of this type of defect in the feature space, facilitating subsequent integration with... The query feature map of the image to be detected is compared with the distance to achieve defect category matching under small sample conditions; the probability response map is a multi-channel probability distribution map obtained by calculating the Euclidean distance between the feature vector of each position in the query feature map of the chassis image and the feature of each defect prototype, and then performing negative exponential transformation and normalization; the defect category map is a single-channel label map of the same size as the original chassis image obtained by upsampling after dynamic semantic attention fusion of the probability response map and the query feature map, and is used to output the final defect category number of each pixel, so that the battery swapping station operators can intuitively understand the precise location and category of the defect, providing a basis for battery swapping.

[0033] In some embodiments, employing dual-sensory attention to suppress background interference in the chassis image and enhance the localization of the protective plate defect target in the defect detection path, the resulting localization feature map of the chassis protective plate defect can be achieved through the following steps: The chassis image is input into the main path of the defect detection path to extract the original feature map; The foreground and background at each spatial location in the original feature map are distinguished, and the response value of the background region is reduced to obtain a background-suppressed discriminative feature map. Determine the spatial correlation matrix between each pixel in the discriminative feature map to obtain the spatial attention weight map of the enhanced protective plate defect target; The spatial attention weight map and all the original feature maps are weighted to obtain the location feature map of the chassis protection plate defect.

[0034] In specific implementation, the chassis image is input into the main path of the defect detection path, and the extraction of the original feature map can be achieved in the following way: the preprocessed chassis image is input into the main network of the defect detection path; this main network shares the same network weight parameters with the main network of the semantic edge path, but the two paths perform forward propagation calculations independently without interfering with each other; then the chassis image first passes through the first convolutional block of the main network, which contains two convolutional layers and one pooling layer, and outputs an image with a resolution reduced to the original. Figure 1A preliminary feature map with half the space resolution and 64 channels is generated, recording local edge and texture information in the image. This preliminary feature map is then passed through the second, third, fourth, and fifth convolutional blocks. Each convolutional block uses a convolution operation with a stride of 2 to progressively halve the spatial resolution while doubling the number of channels. After the fifth convolutional block, an image with spatial resolution reduced to half its original value is obtained. Figure 1 / 32. The original feature map for defect detection with 512 channels integrates multi-level information from shallow to deep layers: shallow channels retain the fine texture and scratch edges of the protective plate surface, while deep channels encode the high-level semantics of the overall structure of the protective plate and the defect area; other embodiments may also use other methods to achieve this, which are not limited here.

[0035] In specific implementation, the foreground and background at each spatial location in the original feature map are distinguished, and the response value of the background region is reduced to obtain the background-suppressed discriminative feature map. This can be achieved in the following way: First, a background attenuation attention module is constructed, which consists of two consecutive convolutional layers: the first convolutional layer uses a 1×1 convolutional kernel to compress the number of channels in the original feature map of defect detection to one-quarter of the original number, and is followed by a batch normalization layer and a linear rectified activation function; the second convolutional layer also uses a 1×1 convolutional kernel to restore the number of channels to the original number, and is followed by a batch normalization layer and a linear rectified activation function; Batch normalization layers and S-curve activation functions are used. The original feature map of defect detection is input into this module. After passing through the above two convolution and activation layers, a foreground confidence map with the same spatial size and number of channels as the input feature map is output. The value of each position in this foreground confidence map is between 0 and 1: the closer the value is to 1, the more likely the chassis image area corresponding to that position is a defective area of ​​the battery pack protection plate; the closer the value is to 0, the more likely it is a defect-free background, such as ground, shadows, or oil stains. Then, the feature vector at each spatial position in the original feature map is multiplied by the value at that position in the foreground confidence map. After multiplying the corresponding values, the feature values ​​of the background region are suppressed to near zero, while the feature values ​​of the defect region can be preserved or even relatively enhanced. The final feature map is the discriminant feature map after background suppression. In this discriminant feature map, there is a spatial correspondence between the high response region and the possible defect location in the chassis image. The spatial attention weight map of the enhanced protective plate defect target can be obtained by determining the spatial correlation matrix between each pixel in the discriminant feature map in the following way: calculate the similarity between all pairs of pixels in the discriminant feature map to form a spatial correlation matrix. Each element in the correlation matrix represents the correlation between two pixel positions in the feature space. Then, for each pixel position, the correlation between the pixel position and all other pixel positions is accumulated to obtain the global attention score of the pixel. After normalizing the scores of all pixels, a two-dimensional weight map with the same width and height as the discriminant feature map is formed, which is the spatial attention weight map. The larger the value in the spatial attention weight map, the stronger the correlation between the region and other regions, and the more likely it is to be the core part of the defect target. Other embodiments can also be implemented in other ways, which are not limited here.

[0036] In specific implementation, the spatial attention weight map and all original feature maps are weighted to obtain the localization feature map of the chassis protection plate defect. This can be achieved by the following method: performing element-wise multiplication between the spatial attention weight map and the original feature map: for each position in the original feature map, its feature vector is multiplied by the weight value of the corresponding position in the spatial attention weight map; the features of regions with high weight values, i.e., regions with strong spatial correlation, are enhanced, while the features of regions with low weight values ​​are suppressed; the feature map obtained after weighting is the localization feature map; the regions with strong responses in this localization feature map correspond to the locations in the chassis image where defects may occur; other methods can also be used in other embodiments, which are not limited here.

[0037] It should be noted that the discriminative feature map in this application is obtained by multiplying the feature vector of each spatial location by the foreground confidence score after inputting the original feature map of defect detection into the background attenuation attention module; the spatial attention weight map is a two-dimensional weight matrix of the same size as the feature map, generated by normalizing the spatial correlation matrix between pixels of the discriminative feature map; the localization feature map is obtained by weighting the spatial attention weight map and the original feature map of defect detection element by element, and is used for subsequent fusion to provide spatial distribution information of defect candidate locations, complement the category information output by the semantic edge path, and realize joint optimization of classification and localization.

[0038] In step 103, the defect category map and the location feature map are fused, and then the fused feature map and the small sample support set are evaluated by introspection metric to obtain the defect evaluation value of the chassis battery pack protection plate. In some embodiments, fusing the defect category map and the location feature map can be achieved by the following steps: The defect category map is compressed using a convolutional layer to align its channel count with the channel count of the localization feature map, and a semantic feature tensor is extracted. The localization feature map and the semantic feature tensor are concatenated along the channel dimension to form a joint feature tensor containing semantic category information and spatial location information; The joint feature tensor is input into a bidirectional feature network, and the interaction of multi-scale features is achieved through bidirectional paths to obtain a fused feature image.

[0039] In specific implementation, aligning the number of channels in the defect category map with the number of channels in the localization feature map through convolutional layers and extracting semantic feature tensors can be achieved in the following way: the defect category map is a three-dimensional feature array with height, width, and number of channels, where the number of channels equals the total number of defect categories, for example, four categories correspond to four channels; the value in each channel represents the probability that the corresponding spatial location belongs to that type of defect; the localization feature map is another three-dimensional feature array with height, width, and number of channels, and its number of channels is usually large, for example, 256 channels; each channel encodes localization information at different levels; since both... Since the number of channels is different, they cannot be directly spliced. Therefore, the defect category map is first expanded by channels. Specifically, a 1×1 convolutional layer is used to perform convolution operation on the defect category map. The number of input channels of this convolutional layer is the number of defect categories, and the number of output channels is set to be the same as the number of channels of the localization feature map. After the 1×1 convolution, a new three-dimensional feature array is obtained, called the semantic feature tensor. The semantic feature tensor has the same height, width and number of channels as the localization feature map. The feature vector at each spatial location is fused with the semantic information of the type of defect at that location. Other embodiments can also be implemented in other ways, which are not limited here.

[0040] In specific implementation, concatenating the localization feature map and the semantic feature tensor along the channel dimension to form a joint feature tensor containing semantic category information and spatial location information can be achieved in the following way: concatenating the semantic feature tensor and the original localization feature map along the channel dimension; for example, assuming the localization feature map has C channels and the semantic feature tensor also has C channels, then the concatenation results in a joint feature tensor with 2C channels; in this joint feature tensor, the first half of the channels retains the original spatial localization response, and the second half of the channels is injected with category semantic information; the joint feature tensor is input into a bidirectional feature network, and multi-scale feature interaction is achieved through bidirectional paths to obtain a fused feature image that can be used. This is achieved by inputting the joint feature tensor into a bidirectional feature pyramid network. This network includes two paths: a top-down path that transmits high-level semantic information to lower levels, enhancing the semantics of small targets; and a bottom-up path that transmits low-level detail information to higher levels, improving the localization of large targets. After several bidirectional interactions, the network outputs a fused feature map with the same size as the input joint feature tensor but with an optimizable number of channels. Each position in this fused feature map incorporates contextual information at different scales. This fused image is used as input for introspection metric evaluation. Other implementation methods are also possible in other embodiments, and are not limited here.

[0041] It should be noted that the joint feature tensor in this application is a three-dimensional feature array with double the number of channels formed by concatenating the localization feature map and the semantic feature tensor in the channel dimension. It is used as the input of the bidirectional feature network, which can carry both spatial location information and semantic category information at the same time, and provide rich feature representation for multi-scale feature interaction.

[0042] In some embodiments, the defect evaluation value of the chassis battery pack protection plate can be obtained by performing introspection metric evaluation on the fused feature map and the small sample support set using the following steps: Flatten the fused feature map into a sequence of feature vectors; Determine the semantic distance between the feature vector sequence and all defect prototype features; Based on all semantic distance and introspective similarity metrics, the confidence level of each feature vector belonging to each defect category is determined, and the defect evaluation value of the chassis battery pack protection plate is obtained.

[0043] In specific implementation, flattening the fused feature map into a sequence of feature vectors can be achieved as follows: the fused feature map is a three-dimensional tensor, assuming the dimensions of the fused feature map are: height H, width W, and number of channels C; the feature vectors of length C at each spatial location in the fused feature map are extracted sequentially and arranged in row-major order into a sequence of length H×W; each element in this sequence is a C-dimensional feature vector, corresponding to a local region in the original chassis image; this sequence of feature vectors is used as the query set for semantic distance calculation, i.e., the feature vector sequence; determining the semantic distance between the feature vector sequence and all defect prototype features can be achieved as follows: for each feature vector in the feature vector sequence, calculate the Euclidean distance between the feature vector and each defect prototype feature; where the smaller the Euclidean distance value, the more similar the image region represented by the feature vector is to the corresponding defect category; for each feature vector, a set of distance values ​​is obtained; other implementation methods can also be used in other embodiments, which are not limited here.

[0044] In practice, the confidence level of each feature vector belonging to each defect category is determined based on all semantic distances and introspective similarity metrics. The defect assessment value of the chassis battery pack protection plate can be obtained in the following way: First, for each feature vector in the fused feature image, the Euclidean distance between the feature vector and each defect prototype vector is calculated to obtain a set of semantic distance values. The smaller the semantic distance value, the more similar the feature vector is to the corresponding defect category. Then, this set of semantic distance values ​​is input into the introspective similarity metric. This introspective similarity metric contains two parallel processing branches: the first branch is the semantic distance branch, which converts the semantic distance values ​​into an initial similarity by calculating the negative exponent of each distance value; the smaller the distance, the closer the initial similarity is to 1. The second branch is the uncertainty branch, which maintains a learnable uncertainty parameter for each defect category. This parameter is minimized during model training by minimizing the classification loss function. Automatic updates are performed, and the numerical value reflects the dispersion of this type of defect in the feature space. The introspective similarity metric function divides the initial similarity output from the first branch by (1 plus the uncertainty parameter output from the second branch) to obtain the corrected confidence score. The physical meaning of this division is that when the uncertainty is large, even if the initial similarity is high, the final confidence score will be suppressed, thereby avoiding the model from making overly confident erroneous judgments in blurred or noisy regions. Next, the confidence scores calculated from all feature vectors in the fused feature image are weighted and averaged, with the weight being the original response intensity corresponding to each feature vector. The weighted average outputs a scalar between 0 and 1, which is used as the defect evaluation value. The closer the defect evaluation value is to 1, the greater the difference between the detection result of the current chassis image and the known defects in the support set, and the more likely it is to be an unknown defect. The closer the defect evaluation value is to 0, the higher the degree of matching with the known defects. Other implementation methods can also be used in other embodiments, which are not limited here.

[0045] It should be noted that the defect assessment value in this application is a scalar between 0 and 1, which is the weighted average of the confidence scores obtained by calculating the introspection similarity metric function of all feature vectors. It is used to determine the degree of matching between the current detection result and the known defects in the support set. This facilitates triggering a rejection signal when the defect assessment value is too high, avoiding the misclassification of unknown defects as known categories and improving the safety of battery swapping station detection.

[0046] In step 104, when the defect assessment value exceeds the rejection threshold, it is determined that the battery pack protection plate of the chassis has an unknown defect and a rejection signal is triggered.

[0047] In some embodiments, when the defect assessment value exceeds the rejection threshold, determining that the chassis battery pack protection plate has an unknown defect and triggering a rejection signal can be achieved through the following steps: Obtain the preset rejection threshold; The defect assessment value is compared with the rejection threshold. If the defect assessment value is greater than the rejection threshold, it is determined that the battery pack protection board has an unknown defect type and a rejection signal is triggered. If the defect evaluation value is less than or equal to the rejection threshold, the defect type is output according to the category with the highest confidence in the defect category map, and the bounding box position of the defect is output according to the localization feature map.

[0048] It should be noted that the reference Figure 3 As shown in the figure, this is a flowchart of the feedback of the rejection signal in some embodiments of this application. The new energy vehicle first enters the battery swapping station, and after the image of the chassis is collected and processed, the rejection signal is finally fed back to the battery swapping station and then transmitted to the new energy vehicle.

[0049] In specific implementation, the defect evaluation value is compared with the rejection threshold. If the defect evaluation value is greater than the rejection threshold, the battery pack protection board is determined to have an unknown defect type, and a rejection signal is triggered. This can be achieved in the following way: After the model training is completed, an independent validation set containing known defects and a small number of artificially simulated unknown defects is used to test the model. The model is run with different thresholds, for example, from 0.05 to 0.95 in increments of 0.05, and the rejection accuracy at each threshold is calculated. The threshold that results in the highest rejection accuracy is selected as the final rejection threshold. This rejection threshold is saved in a configuration file for use during actual detection. The defect evaluation value is then compared with the rejection threshold. If the defect assessment value is greater than the rejection threshold, the battery pack protection plate in the current chassis image is determined to have an unknown defect type, i.e., it does not belong to any known category in the support set. At this time, the system generates a rejection signal, which contains three fields: the timestamp of the current chassis image, the identification number of the battery swapping station, and the specific value of the defect assessment value. This rejection signal is sent as the message content to the control system of the new energy vehicle. The rejection signal is an instruction message containing a timestamp, a battery swapping station identification, and a defect assessment value, generated when the defect assessment value is greater than the rejection threshold. It is sent to the battery swapping station control system to prevent the battery swapping operation and prompt manual review, so as to avoid the damaged battery pack from continuing to be swapped due to model misjudgment and eliminate safety hazards. Other embodiments may also use other methods to achieve this, which are not limited here.

[0050] Furthermore, in another aspect of this application, in some embodiments, this application provides a new energy vehicle chassis defect detection system based on image processing, referring to... Figure 4 The figure is a schematic diagram of the structure of a new energy vehicle chassis defect detection system according to some embodiments of this application. The new energy vehicle chassis defect detection system includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire chassis images of new energy vehicles when they enter the battery swapping station, and to initialize a small sample support set of rare defects in the chassis of new energy vehicles. Processing module 402, in this application, is used to input the chassis image into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the class prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect. It should be noted that the processing module 402 in this application is also used to fuse the defect category map and the location feature map, and then perform introspection metric evaluation on the fused feature map and the small sample support set to obtain the defect evaluation value of the chassis battery pack protection plate. The execution module 403 in this application is mainly used to determine that the battery pack protection board of the chassis has an unknown defect and trigger a rejection signal when the defect evaluation value exceeds the rejection threshold.

[0051] In addition, this application also provides a computer device, which includes a memory and a processor. The memory stores code, and the processor is configured to acquire the code and execute the above-described image processing-based new energy vehicle chassis defect detection method.

[0052] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing an image processing-based method for detecting defects in the chassis of a new energy vehicle, according to some embodiments of this application. The image processing-based method for detecting defects in the chassis of a new energy vehicle in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0053] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0054] The communication bus 502 can be used to transmit information between the aforementioned components.

[0055] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0056] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The method used in the above embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0057] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0058] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0059] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0060] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing-based method for detecting defects in the chassis of new energy vehicles.

[0061] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0062] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting defects in the chassis of new energy vehicles based on image processing, characterized in that, Includes the following steps: Collect chassis images of new energy vehicles when they enter battery swapping stations, and initialize a small sample support set for rare chassis defects of new energy vehicles; The chassis image is input into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the class prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect. The defect category map and the location feature map are fused together, and then the fused feature map and the small sample support set are evaluated by introspection metric to obtain the defect evaluation value of the chassis battery pack protection plate. When the defect assessment value exceeds the rejection threshold, it is determined that the battery pack protection plate of the chassis has an unknown defect and a rejection signal is triggered.

2. The method as described in claim 1, characterized in that, The process involves collecting chassis images of new energy vehicles entering battery swapping stations and initializing a small sample support set for rare chassis defects in new energy vehicles. Specifically, this includes: When a new energy vehicle enters the battery swapping channel of the battery swapping station, it triggers multiple industrial cameras deployed in the pit to take pictures simultaneously. Acquire an image of the area where the battery pack protective plate is located in the chassis of a new energy vehicle, wherein the image at least covers the bottom surface and side edges of the protective plate; The image is subjected to illumination equalization and noise reduction processing to obtain a chassis image; Rare defect samples of labeled battery pack protection panels are selected from historical battery swapping records. Each sample contains a defect image and corresponding defect category label and edge contour annotation, thus forming a small sample support set.

3. The method as described in claim 1, characterized in that, In the semantic edge path, defect prototype features of the small sample support for the battery pack protection plate are extracted based on a class prototype network to obtain a defect category map, specifically including: The defect category set obtained from the small sample support set, each defect category contains at least one labeled defect image of the battery pack protective plate and the corresponding defect edge mask; All defect images are input into the backbone network of semantic edge paths, and support samples are extracted for each defect category; Based on all the defect edge masks, mask average pooling is performed on all the support samples to obtain multiple defect prototype features of the battery pack protection plate. Extract the query feature map of the chassis image, and then determine the semantic distance with each defect prototype feature to obtain the probability response map; The probability response map and the query feature map are dynamically fused using semantic attention to obtain a defect category map of the battery pack protection board.

4. The method as described in claim 1, characterized in that, In the defect detection path, dual-sensor attention is used to suppress background interference in the chassis image and enhance the localization of the protective plate defect target, resulting in a localization feature map of the chassis protective plate defect, specifically including: The chassis image is input into the main path of the defect detection path to extract the original feature map; The foreground and background at each spatial location in the original feature map are distinguished, and the response value of the background region is reduced to obtain a background-suppressed discriminative feature map. Determine the spatial correlation matrix between each pixel in the discriminative feature map to obtain the spatial attention weight map of the enhanced protective plate defect target; The spatial attention weight map and all the original feature maps are weighted to obtain the location feature map of the chassis protection plate defect.

5. The method as described in claim 1, characterized in that, The fusion of the defect category map and the localization feature map specifically includes: The defect category map is compressed using a convolutional layer to align its channel count with the channel count of the localization feature map, and a semantic feature tensor is extracted. The localization feature map and the semantic feature tensor are concatenated along the channel dimension to form a joint feature tensor containing semantic category information and spatial location information; The joint feature tensor is input into a bidirectional feature network, and the interaction of multi-scale features is achieved through bidirectional paths to obtain a fused feature image.

6. The method as described in claim 1, characterized in that, The fused feature map and the small sample support set are evaluated using introspection metrics to obtain the defect evaluation value of the chassis battery pack protection plate, specifically including: Flatten the fused feature map into a sequence of feature vectors; Determine the semantic distance between the feature vector sequence and all defect prototype features; Based on all semantic distance and introspective similarity metrics, the confidence level of each feature vector belonging to each defect category is determined, and the defect evaluation value of the chassis battery pack protection plate is obtained.

7. The method as described in claim 1, characterized in that, When the defect assessment value exceeds the rejection threshold, determining that the chassis battery pack protection plate has an unknown defect and triggering a rejection signal specifically includes: Obtain the preset rejection threshold; The defect assessment value is compared with the rejection threshold. If the defect assessment value is greater than the rejection threshold, it is determined that the battery pack protection board has an unknown defect type and a rejection signal is triggered. If the defect evaluation value is less than or equal to the rejection threshold, the defect type is output according to the category with the highest confidence in the defect category map, and the bounding box position of the defect is output according to the localization feature map.

8. A new energy vehicle chassis defect detection system based on image processing, characterized in that, include: The acquisition module is used to collect chassis images of new energy vehicles when they enter the battery swapping station and initialize a small sample support set of rare defects in the chassis of new energy vehicles. The processing module is used to input the chassis image into a dual-path feature extraction network, thereby extracting the defect prototype features of the small sample support concentrated battery pack protection plate based on the class prototype network in the semantic edge path to obtain a defect category map. In the defect detection path, dual-perception attention is used to suppress background interference in the chassis image and enhance the localization of the protection plate defect target to obtain a localization feature map of the chassis protection plate defect. The processing module is also used to fuse the defect category map and the location feature map, and then perform introspection metric evaluation on the fused feature map and the small sample support set to obtain the defect evaluation value of the chassis battery pack protection plate. The execution module is used to determine that the chassis battery pack protection plate has an unknown defect and trigger a rejection signal when the defect evaluation value exceeds the rejection threshold.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the image processing-based new energy vehicle chassis defect detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the image processing-based new energy vehicle chassis defect detection method as described in any one of claims 1 to 7.

Images