Train body image defect detection method based on multiple models
By employing a multi-model detection method, utilizing Siamese neural networks and YOLOv8 networks for train body image defect detection, the inefficiency and accuracy of traditional detection technologies in complex environments are resolved, achieving rapid and accurate defect detection.
Patent Information
- Application Number
- CN202511157788.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-19
- Publication Date
- 2025-11-18
AI Technical Summary
Traditional train body image detection technology struggles to achieve efficient and accurate defect detection in complex outdoor environments, especially when faced with large amounts of data and variable environments, resulting in insufficient detection efficiency and accuracy.
A multi-model-based detection method is adopted, which uses a Siamese neural network for initial screening and combines it with a YOLOv8 network for localization. An attention mechanism is added and the loss function is optimized to improve detection speed and accuracy.
It enables rapid identification and location of defects in train bodies, improving the accuracy and efficiency of detection, and has strong generalization and rapid processing capabilities.
Smart Images
Figure CN120976185A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent train body detection, specifically to a multi-model-based method for detecting defects in train body images. Background Technology
[0002] In the field of rail transit operation and maintenance technology, visual inspection technology has become a mainstream train monitoring solution to ensure safe train operation. This solution uses visual sensors to collect images of the train's surface, thereby monitoring the condition of the train body, promptly identifying potential defects, and making timely repairs.
[0003] In practical applications, trains operate in complex and ever-changing outdoor environments, resulting in diverse and complex images of the train vehicles. Traditional image detection techniques often struggle to achieve satisfactory detection results. Furthermore, the large volume of vehicle vehicle image data places high demands on the efficiency of the detection algorithms. Therefore, how to efficiently and accurately detect vehicle vehicle defects has become a pressing issue. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a multi-model-based method for detecting defects in train body images. This method addresses the challenges of complex images, large data volumes, and variable environments in train body defect detection by designing a dual-model structure. This enables rapid identification and localization of train body defects, improving the accuracy and efficiency of defect detection. It also features strong generalization ability and fast processing speed.
[0005] The technical solution is as follows:
[0006] A multi-model-based method for detecting defects in train body images involves acquiring images of the train body surface and marking them as images to be inspected.
[0007] The image to be inspected and a pre-stored standard image are input into a pre-trained discriminant network. The discriminant network outputs the probability that the image to be inspected is defective based on the degree of difference between the features of the image to be inspected and the standard image. The standard image is the corresponding defect-free vehicle body surface image.
[0008] If the probability value is less than the preset threshold A, the difference between the features of the image to be tested and the standard image is small, and the image to be tested is recorded as a normal image.
[0009] If the probability value is greater than the preset threshold B, then the difference between the features of the image to be inspected and the standard image is large, and the image to be inspected is recorded as a defect image.
[0010] If the probability value is between threshold A and threshold B, the image to be tested and the standard image are merged into a dual-channel image. The dual-channel image is then scaled and normalized to obtain the processed dual-channel image.
[0011] The processed dual-channel image is input into the localization network, which uses YOLOV8 as the basic framework and includes a ShuffleNet-based backbone network, a C2f-based neck network, and a detection head network connected in sequence.
[0012] The ShuffleNet-based backbone network performs multi-scale feature extraction on the dual-channel image to obtain a multi-scale feature map.
[0013] The neck network based on the C2f module first performs multi-scale feature fusion on the multi-scale feature map output by the backbone network to obtain feature map I, and then uses the C2f module to refine the features of the fused feature map to output feature map II.
[0014] The detection head network includes a decoupling head, which classifies the feature map II, predicts the location of the defect region and the probability value of the presence of the defect, and completes the detection of the defect region in the image to be inspected based on the probability value.
[0015] Furthermore, the ShuffleNet-based backbone network comprises, in sequence: a convolutional layer, a batch normalization layer, an activation function, a max pooling layer, a Stage2 layer, a CoordAtt attention mechanism module, a Stage3 layer, and a Stage4 layer.
[0016] Furthermore, the ShuffleNet-based backbone network performs multi-scale feature extraction on the normalized dual-channel image to obtain the multi-scale feature map as follows:
[0017] The convolutional layer first extracts features from the normalized dual-channel image. The batch normalization layer and activation function normalize and non-linearly activate the convolution result, outputting the first feature map. The max pooling layer downsamples the first feature map to reduce the image size and obtain the second feature map.
[0018] The second feature map is further reduced in size and increased in number of channels by Stage 2 to obtain the third feature map; the CoordAtt attention mechanism module applies coordinate attention weighting to the third feature map according to the attention mechanism to obtain the fourth feature map; the fourth feature map is further reduced in size and increased in number of channels by Stage 3 to obtain the fifth feature map; the fifth feature map is further reduced in size and increased in number of channels by Stage 4 to obtain the sixth feature map.
[0019] Furthermore, the CoordAtt attention mechanism module applies coordinate attention weighting to the third feature map according to the attention mechanism to obtain the fourth feature map, as follows:
[0020] The CoordAtt attention mechanism module performs horizontal global average pooling and vertical global average pooling on the third feature map to obtain feature map F. h Feature map F v For feature map F h Feature map F v The feature map M is obtained by performing dimensionality reduction, activation, and dimensionality increase operations respectively. h Feature map M v ; the feature map M h Feature map M v The dimensions are expanded separately and then added together. The feature maps obtained by addition are activated to generate attention weights. The attention weights are then multiplied element-wise with the third feature map to obtain the enhanced fourth feature map.
[0021] Furthermore, the neck network based on the C2f module first performs multi-scale feature fusion on the multi-scale feature map output by the backbone network to obtain feature map I, as follows:
[0022] The sixth feature map is upsampled and then concatenated with the fourth / fifth feature map along the channel dimension to achieve feature fusion and obtain feature map I.
[0023] or:
[0024] The fifth and sixth feature maps are upsampled and then concatenated with the fourth feature map along the channel dimension to achieve feature fusion and obtain feature map I.
[0025] Furthermore, the discriminant network is a Siamese neural network.
[0026] Furthermore, the loss function of the localization network includes classification loss, bounding box loss, and confidence loss.
[0027] Preferably, weights are incorporated into the calculation of the classification loss, and the calculation method is as follows:
[0028]
[0029] Where N is the specified number of samples; p i To locate the probability of a defective region in the network's predicted output, σ(p) i ) represents p i The value after activation by the activation function; γ is the preset focusing parameter, ranging from 0 to 1;
[0030]
[0031] s is a preset balance coefficient, with a value of 0.6 to 0.1;
[0032]
[0033] Preferably, when training the discrimination network, images of vehicle surfaces with obvious defects and normal vehicle surfaces are used as a set of negative samples, and images of normal vehicle surfaces are used as a set of positive samples; multiple sets of negative samples and multiple sets of positive samples are collected to train the discrimination network.
[0034] Preferably, when training the localization network, the defective vehicle surface image and the normal vehicle surface image are merged into a dual-channel image. The dual-channel image is then scaled and normalized to obtain the processed dual-channel image. This processed image is used as a set of samples, and the coordinates of the defect selection box in the image are recorded using a txt file, which serves as the label.
[0035] Collect multiple sets of samples to train the localization network.
[0036] This method has the following characteristics:
[0037] (1) Using a Siamese neural network as the discriminant network, this network module requires less hardware resources during training and forward inference, making it suitable for vehicle defect detection scenarios characterized by large amounts of image data and fast detection cycles. The discriminant network can quickly screen out images with obvious defects (large defects / large foreign objects), eliminating the need for subsequent detection after the initial screening and accelerating the detection cycle.
[0038] (2) For images to be inspected without obvious defects (small to medium defects), in order to ensure detection accuracy and avoid false detections, the localization network is still built based on YOLOv8n, and the network is reduced from 3 layers to 1 layer (sacrificing the performance of large target detection for the real-time performance of the algorithm), which effectively improves computational efficiency. The CSPDarknet backbone network is replaced with ShuffleNetV2, which reduces the number of parameters and ensures the accuracy of detection.
[0039] (3) This solution further improves the localization network by adding an attention mechanism and optimizing the loss function to improve the accuracy of detection. Attached Figure Description
[0040] Figure 1 This is a schematic diagram of the detection process in a specific implementation method;
[0041] Figure 2 This is a schematic diagram of the ResNet twin neural network structure in a specific implementation. Detailed Implementation
[0042] The technical solution of the present invention will be described in detail below with reference to specific embodiments.
[0043] A multi-model-based method for detecting defects in train body images involves acquiring images of the train body surface and marking them as images to be inspected.
[0044] like Figure 1 As shown, the image to be inspected and the pre-stored standard image are input into a pre-trained discriminant network. The discriminant network outputs the probability that the image to be inspected has a defect based on the degree of difference between the features of the image to be inspected and the standard image; the standard image is the corresponding image of the defect-free vehicle body surface.
[0045] If the probability value is less than the preset threshold A, the difference between the features of the image to be tested and the standard image is small, and the image to be tested is recorded as a normal image.
[0046] If the probability value is greater than the preset threshold B, then the difference between the features of the image to be inspected and the standard image is large, and the image to be inspected is recorded as a defect image.
[0047] If the probability value is between threshold A and threshold B, the image to be tested and the standard image are merged into a dual-channel image. The dual-channel image is then scaled and normalized to obtain the processed dual-channel image.
[0048] The processed dual-channel image is input into the localization network, which uses YOLOV8 as the basic framework. It includes a ShuffleNet-based backbone network, a C2f-based neck network, and a detection head network connected in sequence.
[0049] Multi-scale feature extraction is performed on dual-channel images based on the ShuffleNet backbone network to obtain multi-scale feature maps.
[0050] The neck network based on the C2f module first performs multi-scale feature fusion on the multi-scale feature map output by the backbone network to obtain feature map I. Then, the C2f module is used to refine the features of the fused feature map to output feature map II.
[0051] The detection head network includes a decoupling head, which classifies the feature map II, predicts the location of the defect region and the probability value of the presence of the defect, and completes the detection of the defect region in the image to be inspected based on the probability value.
[0052] In detail, the ShuffleNet backbone network consists of the following layers connected in sequence: convolutional layers, batch normalization layers, activation functions, max pooling layers, Stage 2 layers, CoordAtt attention mechanism module, Stage 3 layers, and Stage 4 layers.
[0053] Multi-scale feature extraction is performed on the normalized dual-channel image based on the ShuffleNet backbone network to obtain the multi-scale feature map as follows:
[0054] The convolutional layer first extracts features from the normalized dual-channel image. The batch normalization layer and activation function normalize and non-linearly activate the convolution result, outputting the first feature map. The max pooling layer downsamples the first feature map to reduce the image size and obtain the second feature map.
[0055] The second feature map is further reduced in size and increased in number of channels by Stage 2 to obtain the third feature map; the CoordAtt attention mechanism module applies coordinate attention weighting to the third feature map according to the attention mechanism to obtain the fourth feature map; the fourth feature map is further reduced in size and increased in number of channels by Stage 3 to obtain the fifth feature map; the fifth feature map is further reduced in size and increased in number of channels by Stage 4 to obtain the sixth feature map.
[0056] Specifically, the CoordAtt attention mechanism module applies coordinate attention weighting to the third feature map based on the attention mechanism to obtain the fourth feature map, as follows:
[0057] The CoordAtt attention mechanism module performs horizontal global average pooling and vertical global average pooling on the third feature map to obtain feature map F. h Feature map F v For feature map F h Feature map F v The feature map M is obtained by performing dimensionality reduction, activation, and dimensionality increase operations respectively. h Feature map M v ; the feature map M h Feature map M v The dimensions are expanded separately and then added together. The feature maps obtained by addition are activated to generate attention weights. The attention weights are then multiplied element-wise with the third feature map to obtain the enhanced fourth feature map.
[0058] More specifically, the neck network based on the C2f module first performs multi-scale feature fusion on the multi-scale feature maps output by the backbone network to obtain feature map I, as follows:
[0059] The sixth feature map is upsampled and then concatenated with the fourth / fifth feature map along the channel dimension to achieve feature fusion and obtain feature map I.
[0060] or:
[0061] The fifth and sixth feature maps are upsampled and then concatenated with the fourth feature map along the channel dimension to achieve feature fusion and obtain feature map I.
[0062] In this scheme, the discriminant network is a Siamese neural network. The loss function of the localization network includes classification loss, bounding box loss, and confidence loss.
[0063] To improve detection accuracy, as a preferred implementation method, weights are added to the calculation of the classification loss, and the calculation method is as follows:
[0064]
[0065] Where N is the specified number of samples; p i To locate the probability of a defective region in the network's predicted output, σ(p) i ) represents p i The value after activation by the activation function; γ is the preset focusing parameter, ranging from 0 to 1;
[0066]
[0067] s is a preset balance coefficient, with a value of 0.6 to 0.1;
[0068]
[0069] To facilitate understanding of the technical solution of the present invention, the following exemplary description is provided:
[0070] First, the model training process is performed:
[0071] 1) Training the discriminant network:
[0072] Images of vehicle surfaces with obvious anomalies (images to be inspected) and images of normal vehicle surfaces (standard images) are used as a group of negative samples. Obvious anomalies can be understood as those that can be effectively distinguished by the naked eye, or where the abnormal area accounts for more than 30% of the entire image; the label of this sample is 0.
[0073] The normal vehicle body surface image (the image to be inspected) and the normal vehicle body surface image (the standard image) are used as a group of positive samples, and the label of this sample is 1;
[0074] Collect multiple sets of negative samples and multiple sets of positive samples to train a discriminant network, which is a Siamese neural network. The backbone network of the Siamese neural network can be a mature network structure, such as the VGG series, Inception, DenseNet, EfficienNet, ResNet, etc.
[0075] Taking the ResNet twinned neural network architecture as an example:
[0076] To improve training speed, the ResNet network was pre-trained using other image datasets. After training, the ResNet network was used to build a ResNet Siamese neural network.
[0077] When training with vehicle images, only the layer 4 and fully connected (fc) layers of the ResNet network need to be updated, while all other layers are frozen.
[0078] like Figure 2 A set of images from the sample is simultaneously input into a ResNet Siamese network structure to extract features, resulting in two feature vectors (e.g., 512-dimensional and 1024-dimensional). These two high-dimensional feature vectors are then passed through two fully connected layers to obtain two dimensionality-reduced feature vectors (e.g., 50-dimensional and 20-dimensional). A contrastive loss function is calculated on the dimensionality-reduced feature vectors to obtain the degree of difference (feature distance) between the features of the two images. Backpropagation is performed using the Adam optimizer to update the weights. After multiple rounds of training, the converged weights are obtained. The model parameters of the discriminant network are stored.
[0079] 2) Training the localization network:
[0080] Images of vehicle surfaces with defects (such as foreign objects, oil leaks, or missing parts) are merged with normal vehicle surface images into a dual-channel image. The dual-channel image is then scaled to meet the size required for model input (e.g., 640 pixels × 640 pixels). Normalization is then performed (each grayscale value is divided by 255 pixel by pixel) to obtain a normalized dual-channel image. This normalized image is used as a sample, and the coordinates of the center point of the defect selection box in the image, as well as the length and height of the selection box, are recorded in a txt file as labels.
[0081] Collect multiple sets of samples to train the localization network.
[0082] During training, the sample and label information (txt file) are input into the localization network. The localization network outputs the bounding box coordinates (center coordinates), length, height, and confidence score (probability of the presence of a defect category) of the defect region.
[0083] The loss function is calculated, which consists of three parts: classification loss L. focal Bounding box loss L CIou and confidence loss L obj ;
[0084] In this embodiment, because the defective region accounts for a small proportion of the entire image to be inspected, the localization network is easily dominated by a large number of normal regions, ignoring defect features. Therefore, the classification loss L is adjusted. focal Apply weight k·(1-p′) γ This reduces the impact of normal regions and improves the model's sensitivity to defects.
[0085] Specifically:
[0086]
[0087] Where N is the specified number of samples; for example, if N = 16, then L is calculated once every 16 samples. focal ; i represents the i-th sample group;
[0088] p i To determine the probability that the i-th sample in the network's predicted output contains a defect region, σ(p) i ) represents p i The value after activation by the activation function; γ is the preset focusing parameter, ranging from 0 to 1;
[0089]
[0090] s is a balance coefficient, with a value of 0.6 to 0.1. Since the normal area is usually larger than the defect area in an image, s is set to a value of 0.6 to 0.1 to increase the weight of the defect area.
[0091]
[0092] In this embodiment, the threshold is 60% to 80%.
[0093] The overlap between the predicted bounding box and the actual bounding box of the label can be calculated using the intersection-union ratio of the two bounding boxes.
[0094] Explanatory, (1-p′) γ Dynamic weighting adjustment for the "difficulty" of samples: e.g., when p i When it is greater than 0.8, or p i When the value is less than 0.2, the output of the localization network is relatively certain, and at this time (1-p′) γ It will be relatively small, close to 0; when p i When the value is between 0.2 and 0.8, the output of the localization network is relatively uncertain, for example, p i =0.5, at this point the localization network considers there to be a 50% probability of a defect. Therefore, the localization network considers the existence of a defect to be uncertain, and at this point (1-p′) γ This will be relatively large, increasing the proportion of this type of sample, making the localization network pay more attention to this type of sample, and improving the recognition ability.
[0095] Furthermore, in this embodiment, the bounding box loss L CIou and confidence loss L obj Using the typical calculation method of YOLO V8, the calculation formula is as follows:
[0096]
[0097] Wherein, IoU is the intersection-union ratio (the ratio between the intersection and union of the areas of the predicted bounding boxes of the defect regions predicted by the localization network and the actual bounding boxes of the defect regions recorded by the sample labels); ρ 2 The squared Euclidean distance between the predicted selection box and the center point of the actual selection box; c 2 To compare the predicted selection box with the actual selection box, the square of the diagonal length of the smaller selection box is calculated. w and h are the width and height of the predicted selection box, respectively. gt h gt α represents the width and height of the actual selection box; α is the balance coefficient.
[0098]
[0099] Where M is the total number of predicted selection boxes obtained by the localization network after predicting N sets of samples;
[0100]
[0101] The confidence level (probability of the existence of a defect category) of the j-th predicted bounding box by the model. This represents the confidence level after activation by the activation function.
[0102] The Adam optimizer is used for backpropagation to update the weights. After multiple rounds of training, the converged weights are obtained; the model parameters are then stored.
[0103] In practice, there are no special restrictions on the use of activation functions. Common activation functions such as ReLU, sigmoid, tanh, and ELU can be used.
[0104] In order to ensure that the model's generalization ability and accuracy meet the requirements, the number of samples should generally be no less than 500, and more preferably 5,000 to 20,000.
[0105] After the model training is completed, the following is the formal testing process:
[0106] Acquire images of the vehicle body surface and mark them as images to be inspected;
[0107] The image to be inspected and a pre-stored standard image are input into a trained discrimination network. The discrimination network outputs the probability that the image to be inspected has a defect based on the degree of difference between the image to be inspected and the standard image (features of the two images). The standard image is the corresponding image of the defect-free vehicle body surface. For example, the image to be inspected is an image of a train door (with a defect); the standard image is an image of the door in a normal state.
[0108] If the probability value is less than the preset threshold A (ranging from 0.1 to 0.3), the difference between the image to be tested and the standard image (features of the two images) is small, and the image to be tested is recorded as a normal image.
[0109] If the probability value is greater than the preset threshold B (ranging from 0.5 to 0.8), then the difference between the image to be inspected and the standard image (features of the two images) is large, and the image to be inspected is recorded as a defective image.
[0110] If the probability value is between threshold A and threshold B, the image to be tested and the standard image are merged into a dual-channel image. The dual-channel image is then scaled (to meet the size required for the model input) and normalized (in this image, a single pixel contains two gray values, and each gray value is divided by 255 pixel by pixel) to obtain the processed dual-channel image (2×H×W), where H and W represent the height and width of the image, respectively.
[0111] The processed dual-channel image is input into the localization network, which uses YOLOV8 as the basic framework. It includes a ShuffleNet-based backbone network, a C2f-based neck network, and a detection head network.
[0112] The ShuffleNet-based backbone network extracts multi-scale features from the normalized dual-channel image to obtain multi-scale feature maps. Specifically, the ShuffleNet-based backbone network consists of the following layers connected in sequence: convolutional layer, batch normalization layer, activation functions (ReLU, sigmoid, tanh, ELU), max pooling layer, Stage 2 layer, CoordAtt attention mechanism module, Stage 3 layer, and Stage 4 layer.
[0113] The convolutional layer first extracts features from the normalized dual-channel image. The batch normalization layer and activation function normalize and non-linearly activate the convolution result, outputting the first feature map (C1×H1×W1), where C1, H1, and W1 represent the number of channels, height, and width of the image, respectively, such as 24×640×640. The max pooling layer downsamples the first feature map to reduce the image size, resulting in the second feature map (C1×H2×W2). Typically, C1 takes values from 12 to 36, H1 = 1 / 2H, W1 = 1 / 2W, H2 = 1 / 4H, and W2 = 1 / 4W.
[0114] The second feature map is further reduced in size and increased in number of channels by the Stage2 layer to obtain the third feature map (C3×H3×W3);
[0115] In this embodiment, taking scaling factor = 1 and C3 = 116 as an example; H3 = 1 / 2H2 and W3 = 1 / 2W2;
[0116] Interpretively, compared to the smaller feature maps output by Stage 3 and Stage 4, the feature maps output by Stage 2 retain richer spatial details and already possess some semantic feature extraction capabilities. Adding a cross-scale attention module at this point effectively fuses contextual information from different scales onto the medium-scale feature maps. This avoids the limited effectiveness of the attention mechanism in the early feature extraction stage due to insufficient semantic information, and also prevents the attention from affecting the accurate localization of defects in Stages 3 and 4 due to the small feature map size and significant loss of spatial details.
[0117] Therefore, as a preferred implementation, in this embodiment, a CoordAtt attention mechanism module is added between Stage2 and Stage3. The CoordAtt attention mechanism module performs coordinate attention weighting on the third feature map according to the attention mechanism to obtain the fourth feature map (C4×H4×W4); specifically:
[0118] The CoordAtt attention mechanism module performs horizontal global average pooling and vertical global average pooling on the third feature map to obtain feature map F. h (C3×1×W3), Feature Map F v (C3×H3×1), for feature map F h Feature map F v The feature map M is obtained by performing dimensionality reduction (reducing the number of channels to 1 / 5C3 to 1 / 10C3), activation, and dimensionality increase (restoring the number of channels to C3). h Feature map M v ;
[0119] Feature map M h Feature map M v Expand the dimensions (feature map M) respectively h The height is restored to H3, and the feature map M v The width of the feature map is restored to W3, and then the two maps are added together. The feature map obtained by addition is activated (the activation function can be any one of ReLU, sigmoid, tanh, or ELU) to generate attention weights. The attention weights are then multiplied element-wise with the third feature map to obtain the enhanced fourth feature map (C4×H4×W4), where, numerically, C4=C3, H4=H3, and W4=W3.
[0120] The fourth feature map is further reduced in image size and increased in number of channels by Stage 3 to obtain the fifth feature map (C5×H5×W5), where C5=2×C4, H5=1 / 2H4, W5=1 / 2W4; the fifth feature map is further reduced in image size and increased in number of channels by Stage 4 to obtain the sixth feature map (C6×H6×W6), where C6=2×C5, H6=1 / 2H5, W6=1 / 2W5.
[0121] Next, the neck network based on the C2f module performs multi-scale feature fusion on the multi-scale feature map output by the backbone network to obtain feature map I, as follows:
[0122] The sixth feature map is upsampled and then concatenated with the fourth / fifth feature map along the channel dimension to achieve feature fusion, resulting in feature map I((C6+C4 / C5)×H7×W7); at this time, H7=H4 / H5 and W7=W4 / W5.
[0123] or:
[0124] The fifth and sixth feature maps are upsampled and then concatenated with the fourth feature map along the channel dimension to achieve feature fusion, resulting in feature map I ((C6+C4+C5)×H7×W7). At this time, H7=H4 and W7=W4.
[0125] The C2f module (one 1*1 convolutional layer, multiple bottleneck layers and another 1*1 convolutional layer) is then used to refine the features of the fused feature map, outputting feature map II (C8×H8×W8), where C8 takes values of 96 to 256, H8 takes values of 20 to 80, and W8 takes values of 20 to 80.
[0126] Finally, the detection head network includes a decoupling head that classifies feature map II, predicts the location of defect regions and the probability of their presence, and then detects defect regions in the image based on the probability values. If the probability value is greater than a threshold, a defect is considered to exist; otherwise, it is considered normal.
[0127] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.
Claims
1. A method for detecting defects in train body images based on multiple models, characterized in that, Acquire images of the vehicle body surface and mark them as images to be inspected; The image to be inspected and a pre-stored standard image are input into a pre-trained discriminant network. The discriminant network outputs the probability that the image to be inspected is defective based on the degree of difference between the features of the image to be inspected and the standard image. The standard image is the corresponding defect-free vehicle body surface image. If the probability value is less than the preset threshold A, the difference between the features of the image to be tested and the standard image is small, and the image to be tested is recorded as a normal image. If the probability value is greater than the preset threshold B, then the difference between the features of the image to be inspected and the standard image is large, and the image to be inspected is recorded as a defect image. If the probability value is between threshold A and threshold B, the image to be tested and the standard image are merged into a dual-channel image. The dual-channel image is then scaled and normalized to obtain the processed dual-channel image. The processed dual-channel image is input into the localization network, which uses YOLOV8 as the basic framework and includes a ShuffleNet-based backbone network, a C2f-based neck network, and a detection head network connected in sequence. The ShuffleNet-based backbone network performs multi-scale feature extraction on the dual-channel image to obtain a multi-scale feature map. The neck network based on the C2f module first performs multi-scale feature fusion on the multi-scale feature map output by the backbone network to obtain feature map I, and then uses the C2f module to refine the features of the fused feature map to output feature map II. The detection head network includes a decoupling head, which classifies the feature map II, predicts the location of the defect region and the probability value of the presence of the defect, and completes the detection of the defect region in the image to be inspected based on the probability value.
2. The train body image defect detection method based on multiple models as described in claim 1, characterized in that: The ShuffleNet-based backbone network comprises, in sequence: convolutional layer, batch normalization layer, activation function, max pooling layer, Stage 2 layer, CoordAtt attention mechanism module, Stage 3 layer, and Stage 4 layer.
3. The train body image defect detection method based on multiple models as described in claim 2, characterized in that: The ShuffleNet-based backbone network performs multi-scale feature extraction on the normalized dual-channel image to obtain the multi-scale feature map as follows: The convolutional layer first extracts features from the normalized dual-channel image. The batch normalization layer and activation function normalize and non-linearly activate the convolution result, outputting the first feature map. The max pooling layer downsamples the first feature map to reduce the image size and obtain the second feature map. The second feature map is further reduced in size and increased in number of channels by Stage 2 to obtain the third feature map; the CoordAtt attention mechanism module applies coordinate attention weighting to the third feature map according to the attention mechanism to obtain the fourth feature map; the fourth feature map is further reduced in size and increased in number of channels by Stage 3 to obtain the fifth feature map; the fifth feature map is further reduced in size and increased in number of channels by Stage 4 to obtain the sixth feature map.
4. The train body image defect detection method based on multiple models as described in claim 3, characterized in that: The CoordAtt attention mechanism module applies coordinate attention weighting to the third feature map based on the attention mechanism to obtain the fourth feature map, as follows: The CoordAtt attention mechanism module performs horizontal global average pooling and vertical global average pooling on the third feature map to obtain feature map F. h Feature map F v For feature map F h Feature map F v The feature map M is obtained by performing dimensionality reduction, activation, and dimensionality increase operations respectively. h Feature map M v ; the feature map M h Feature map M v The dimensions are expanded separately and then added together. The feature maps obtained by addition are activated to generate attention weights. The attention weights are then multiplied element-wise with the third feature map to obtain the enhanced fourth feature map.
5. The train body image defect detection method based on multiple models as described in claim 3, characterized in that: The neck network based on the C2f module first performs multi-scale feature fusion on the multi-scale feature map output by the backbone network to obtain feature map I, as follows: The sixth feature map is upsampled and then concatenated with the fourth / fifth feature map along the channel dimension to achieve feature fusion and obtain feature map I. or: The fifth and sixth feature maps are upsampled and then concatenated with the fourth feature map along the channel dimension to achieve feature fusion and obtain feature map I.
6. The train body image defect detection method based on multiple models as described in claim 1, characterized in that: The discriminant network is a twin neural network.
7. The train body image defect detection method based on multiple models as described in claim 1, characterized in that: The loss function of the localization network includes classification loss, bounding box loss, and confidence loss.
8. The train body image defect detection method based on multiple models as described in claim 1, characterized in that: Weights are added during the calculation of the classification loss, and the calculation method is as follows: Where N is the specified number of samples; p i To locate the probability of a defective region in the network's predicted output, σ(p) i ) represents p i The value after activation by the activation function; γ is the preset focusing parameter, ranging from 0 to 1; s is a preset balance coefficient, with a value of 0.6 to 0.1; 9. The train body image defect detection method based on multiple models as described in claim 1, characterized in that: When training the discrimination network, images of vehicle surfaces with obvious defects and normal vehicle surfaces are used as a set of negative samples, and images of normal vehicle surfaces are used as a set of positive samples. Multiple sets of negative samples and multiple sets of positive samples are collected to train the discrimination network.
10. The train body image defect detection method based on multiple models as described in claim 1, characterized in that: When training the localization network, the defective vehicle surface image and the normal vehicle surface image are merged into a dual-channel image. The dual-channel image is then scaled and normalized to obtain the processed dual-channel image. This processed image is used as a set of samples, and the coordinates of the defect selection box in the image are recorded in a txt file as the label. Collect multiple sets of samples to train the localization network.