Shell defect detection method and device, electronic equipment and storage medium
By using a fusion network and feature fusion module in shell defect detection, combined with bounding box loss optimization training, the problems of low detection accuracy and slow speed in the prior art are solved, and more efficient shell defect recognition is achieved.
Patent Information
- Application Number
- CN202411200261.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-27
AI Technical Summary
In the prior art, the accuracy of shell defect detection is low and the detection speed is slow, especially when dealing with complex defects and different lighting conditions, it is difficult to effectively identify them.
A shell defect detection method is adopted. By obtaining the shell image to be detected and inputting it into the trained defect detection model, the fusion network, feature extraction infrastructure and Detect module are used for detection. The feature fusion module fuses different convolution kernel sizes. Feature maps and trained in the second half of the training using samples with bounding box losses greater than the average, combining attention modules and loss function optimization models.
It improves the accuracy and speed of shell defect detection, can effectively identify complex defects and adapt to different light source conditions, and reduces computing resource consumption and detection time.
Smart Images
Figure CN119205639A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image detection technology, and in particular to a shell defect detection method, device, electronic equipment and storage medium. Background Art
[0002] Shell defect detection refers to the process of detecting whether there are defects on the outer surface of a product by using technical methods. These defects may be cracks, scratches, wear, deformation, etc. The defects of the product shell will affect the appearance, and may also weaken the structure of the product, affect its performance and life, and even bring safety hazards. The shells of some electronic products are prone to defects. For example, the shells of laptops are prone to defects in various forms and scales. The existing technology uses image processing to detect shell defects, but for some defects with more complex scales or forms, even if the number of training samples of the detection model is not small, it is difficult to effectively converge during the training process, resulting in low detection accuracy; in addition, the imaging effect of a few defects under the existing lighting conditions is not ideal, and it is necessary to image clearly under different lighting conditions to better identify these defects, but it also increases resource consumption and time overhead, and reduces the overall detection speed. Summary of the invention
[0003] Based on this, the present invention provides a shell defect detection method, device, electronic device and storage medium to solve the technical problems of low accuracy and slow detection speed of shell defect detection in the prior art.
[0004] The technical solution of the present invention is as follows: a shell defect detection method is provided, comprising: obtaining a shell image to be detected;
[0005] The shell image to be detected is input into the trained defect detection model to obtain the shell defect detection result. The defect detection model is obtained by training the initial defect detection model using the shell image samples to be detected. The initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect. The fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss.
[0006] Further, the shell image samples to be detected include main light source image samples and auxiliary light source image samples; accordingly, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, including:
[0007] The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module to obtain a fused feature map. The fused feature map enters the feature reconstruction module, and the feature reconstruction module introduces an attention module.
[0008] Furthermore, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, including:
[0009] The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module according to a fusion formula, and the fusion formula includes
[0010]
[0011] Among them, f o (x,y) represents the fused feature map, Represents the pixel with coordinates (x, y) in the jth channel of the ith image sample, c is the total number of channels, the first image sample is the main light source image sample or the auxiliary light source image sample, the second image sample is the auxiliary light source image sample or the main light source image sample, fuse represents the element fusion rule, and the element fusion rule is to take the maximum value, minimum value or average value of each element in the brackets.
[0012] Furthermore, the shell defect detection method further includes:
[0013] When the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss, the corresponding total loss is obtained according to the classification loss, the bounding box loss, the confidence loss and the total loss calculation formula. The total loss function calculation formula includes
[0014]
[0015] Among them, Loss total_>GIoU is the total loss, 1-GIoU k is the bounding box loss of the kth shell image sample to be detected, E(1-GIoU) is the average bounding box loss, n is the total number of shell image samples to be detected, and loss cls is the classification loss, loss obj is the bounding box loss, loss rect is the confidence loss.
[0016] Furthermore, the feature extraction infrastructure includes at least one feature extraction module among a Focus module, a Conv module, a C3 module and a spatial pyramid pooling module.
[0017] Furthermore, the Head includes an FPN module and a PAN module.
[0018] Furthermore, the Detect includes a convolutional layer.
[0019] Another technical solution of the present invention is as follows: a housing defect detection device is also provided, comprising a data acquisition module and a detection module;
[0020] The data acquisition module is used to acquire the image of the shell to be detected;
[0021] The detection module is used to input the shell image to be detected into the trained defect detection model to obtain the shell defect detection result. The defect detection model is obtained by training the initial defect detection model using the shell image samples to be detected. The initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect. The fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss.
[0022] Another technical solution of the present invention is as follows: an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the shell defect detection method described in any of the above technical solutions is implemented.
[0023] Another technical solution of the present invention is as follows: a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the shell defect detection method as described in any of the above technical solutions is implemented.
[0024] The beneficial effects of the present invention are: obtaining a shell image to be detected; inputting the shell image to be detected into a trained defect detection model to obtain a shell defect detection result, wherein the defect detection model is obtained by training an initial defect detection model using a shell image sample to be detected, and the initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect, and the fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses multiple feature maps of different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using a shell image sample to be detected whose bounding box loss is greater than the average bounding box loss. Through the above technical solution, the detection of shell defects can be achieved, and the accuracy and speed of defect detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 A schematic diagram of a process flow of a shell defect detection method provided by an embodiment of the present invention;
[0026] Figure 2 A schematic diagram of the structure of a fusion network provided in an embodiment of the present invention;
[0027] Figure 3 A schematic diagram of the structure of a housing defect detection device provided by an embodiment of the present invention;
[0028] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;
[0029] Figure 5 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0030] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. The preferred embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.
[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0032] Figure 1 FIG. 1 is a flow chart of a shell defect detection method according to an embodiment of the present invention. It should be noted that if there is substantially the same result, the shell defect detection method of the present invention is not limited to the above two methods. Figure 1 The process sequence shown is limited. Figure 1 As shown, the shell defect detection method comprises the following steps:
[0033] S101, obtaining an image of a housing to be inspected;
[0034] In a specific embodiment, an AOI (Automated Optical Inspection) device or other camera equipment may be used to capture images under multiple light sources and multiple angles to obtain an image of the shell to be detected. For example, an AOI device may be used to capture images under two light sources and two corresponding angles to obtain a main light source image and an auxiliary light source image. The main light source image and the auxiliary light source image may be used to form an image of the shell to be detected. The main light source image and the auxiliary light source image may be preprocessed to form the image of the shell to be detected.
[0035] S102, input the shell image to be detected into the trained defect detection model to obtain the shell defect detection result, the defect detection model is obtained by training the initial defect detection model with the shell image samples to be detected, the initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect, the fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module, during the training process, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, and in the second half of the training, the initial defect detection model is trained with the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss.
[0036] In a specific embodiment, similar to obtaining the image of the shell to be detected, images can be taken under multiple different light sources and multiple different angles by a camera device such as an AOI (Automated Optical Inspection) device, and the image can be preprocessed to obtain a sample of the shell image to be detected, and each sample of the shell image to be detected can include a sample of the main light source image and a sample of the auxiliary light source image. The training process can be performed on a server, and data enhancement methods such as Mosaic can be used to diversify the samples, and the anchor frame generation algorithm can be used to update the anchor frame to improve the detection speed and reduce missed detection. The above-mentioned initial defect detection model is trained using the sample of the shell image to be detected, and the feature information of the image under different light sources is integrated using a fusion network, and the feature representation of defects under various lighting conditions is learned to realize defect detection for imaging with different light sources. The feature extractor extracts the features of the image layer by layer, generates a multi-scale feature map, captures the contextual information of different scales, and improves the model's ability to understand and distinguish the target. Finally, the feature map of each scale is used for target detection, and for each scale, the anchor frame is used to predict the location and category of the target.
[0037] In a specific embodiment, the structural diagram of the fusion network is as follows: Figure 2 As shown, the fusion network includes a feature extraction module (Feature Extraction module), a feature fusion module (Feature Fusion module) and a feature reconstruction module (Feature Reconstruction module), and the output of the fusion network can enter the feature extraction infrastructure (Backbone structure) after being processed by the ReLU function.
[0038] In some embodiments, the shell image samples to be detected include main light source image samples and auxiliary light source image samples; accordingly, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, including:
[0039] The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module to obtain a fused feature map. The fused feature map enters the feature reconstruction module, and the feature reconstruction module introduces an attention module.
[0040] In a specific embodiment, multi-light source image input is used for feature-level fusion, such as Figure 2 As shown in the figure, the main light source image samples and the auxiliary light source image samples pass through the feature extraction module to generate multiple feature maps with different convolution kernel sizes. These feature maps are sent to the feature fusion module, which fuses the features through a Fuse operation. The fused feature maps enter the feature reconstruction module. The feature reconstruction module introduces the attention module, which can retain the features of the main light source image samples while supplementing the unique features of the auxiliary light source image samples. Figure 2 As shown, the feature extraction module includes 1×1CONV and 3×3CONV, and the feature reconstruction involves 1×1CONV, Avgpool (pooling layer) and sigmoid (activation function).
[0041] In some embodiments, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, including:
[0042] The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module according to a fusion formula, and the fusion formula includes
[0043]
[0044] Among them, f o (x,y) represents the fused feature map, Represents the pixel with coordinates (x, y) in the jth channel of the ith image sample, c is the total number of channels, the first image sample is the main light source image sample or the auxiliary light source image sample, the second image sample is the auxiliary light source image sample or the main light source image sample, fuse represents the element fusion rule, and the element fusion rule is to take the maximum value, minimum value or average value of each element in the brackets.
[0045] In an embodiment of the present invention, a feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module according to a fusion formula, thereby integrating feature information under different light sources, learning the feature representation of defects under various lighting conditions, and realizing defect detection for imaging with different light sources, thereby improving the accuracy of defect detection.
[0046] In some embodiments, the housing defect detection method further includes:
[0047] When the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss, the corresponding total loss is obtained according to the classification loss, the bounding box loss, the confidence loss and the total loss calculation formula. The total loss function calculation formula includes
[0048]
[0049] Among them, Loss total_>GIoU is the total loss, 1-GIoU k is the bounding box loss of the kth shell image sample to be detected, E(1-GIoU) is the average bounding box loss, n is the total number of shell image samples to be detected, and loss cls is the classification loss, loss obj is the bounding box loss, loss rect is the confidence loss.
[0050] In a specific embodiment, the second half of the training can be a training process in which the total loss corresponding to the model remains unchanged after continuous training for multiple times (for example, 20 times). From the first half of the training to the second half of the training, the total loss gradually converges but there are still defects that are difficult to converge; by using the shell image samples to be detected whose bounding box loss is greater than the average value of the bounding box loss to train the initial defect detection model in the second half of the training, and using the total loss corresponding to the shell image samples to be detected whose bounding box loss is greater than the average value of the bounding box loss as the subsequent total loss to continue training the model, so that the loss converges, the feature capture of difficult-to-detect defects is strengthened, and the training bias for difficult-to-detect samples is increased, thereby enhancing the training effect. The total loss corresponding to the second half of the above training is obtained according to the classification loss, bounding box loss, confidence loss and total loss calculation formula between the prediction box and the annotation box. It should be noted that the total loss corresponding to the first half of the training can also be the sum of the classification loss, the bounding box loss and the confidence loss, but in the first half of the training, the training samples are all shell image samples to be detected.
[0051] In a specific embodiment, after calculating the classification loss, bounding box loss and confidence loss, the total loss is determined, and the weight of the model is updated through the back-propagation algorithm and the gradient of the loss function, and the loss function is minimized to optimize the model and improve the accuracy. In the second half of the training, the loss no longer decreases and the model weight tends to be stable. The initial defect detection model can continue to be trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss, until the loss no longer decreases again, the model weight tends to be stable, and the update stops. As an example, the pseudo code for the calculation of the total loss and back-propagation is as follows: recent_losses = [] # used to store the loss values of the last few times
[0052] loss_threshold = lossR #convergence threshold
[0053] num_recent = numR # The number of recent loss values used to determine convergence
[0054] hard_sample_indices = [] # used to store the numbers of difficult samples
[0055] #Training loop
[0056] for epoch in range(num_epochs):
[0057] model.train()
[0058] #Calculate the loss of the current batch
[0059] total_loss, batch_losses = calculate_loss(model, train_loader, criterion, use_hard_samples=(len(hard_sample_indices) > 0), hard_sample_indices=hard_sample_indices)
[0060] # Update recent losses
[0061] recent_losses.append(total_loss)
[0062] if len(recent_losses) > num_recent:
[0063] recent_losses.pop(0)
[0064] # Check if the loss has converged
[0065] if has_converged(recent_losses, loss_threshold):
[0066] mean_loss = sum(batch_losses) / len(batch_losses)
[0067] hard_sample_indices = [i for i, loss in enumerate(batch_losses) if loss > mean_loss]
[0068] print(f"Epoch {epoch}: Using hard samples for training with {len(hard_sample_indices)} samples.")
[0069] else:
[0070] hard_sample_indices = []
[0071] print(f"Epoch {epoch}: Total loss = {total_loss}")
[0072] # Calculate the total loss of high-loss samples
[0073] if high_loss_indices:
[0074] total_loss=compute_total_loss(all_samples,high_loss_indices)
[0075] #Backward Propagation and Optimization
[0076] optimizer.zero_grad()
[0077] total_loss.backward()
[0078] optimizer.step().
[0079] In a specific embodiment, after model training, the model can be tested on a machine in a factory workshop. The testing process can include deploying the model with updated weights to the workshop target detection equipment, using the shell image samples of defective products to continuously detect in the equipment, and testing whether the model training weights are valid. If valid, they are used for defect detection. If not, the model weights are continuously optimized. The formal defect detection process can also be carried out on a machine in a factory workshop, using a trained defect detection model (which can be a tested defect detection model) to perform defect detection, output the upper left corner coordinates and height and width of all detected defects, post-process all detected defects, and manually re-judge the defects of defective products.
[0080] In some embodiments, the feature extraction infrastructure includes at least one feature extraction module among a Focus module, a Conv module, a C3 module, and a spatial pyramid pooling module.
[0081] In a specific embodiment, the C3 module may be a residual structure formed by a structure such as Conv (convolution), which may increase the depth of the neural network.
[0082] In some embodiments, the Head includes an FPN module and a PAN module.
[0083] In a specific embodiment, the Head part includes an FPN module and a PAN module. The FPN module is top-down and obtains a feature map for prediction by fusing high-level features through upsampling and low-level features. The FPN module (layer) conveys strong semantic features from top to bottom, while the PAN module conveys strong positioning features from bottom to top. The FPN module transmits deep semantic features to shallow layers, thereby enhancing semantic expression at multiple scales, while the PAN module transmits shallow positioning information to deep layers, thereby enhancing positioning capabilities at multiple scales.
[0084] In some embodiments, the Detect comprises a convolutional layer.
[0085] In a specific embodiment, Detect includes convolution (convolutional layer), which can convert the output of the previous part (ie, Head) into the location and category probability of the target box.
[0086] In a specific embodiment, an industrial camera is used in a workshop to capture defect images of industrial targets at various angles and light sources, and the images are annotated by labelimg. Data enhancement methods such as Mosaic can be used to diversify the training samples. The histogram equalization algorithm is used to enhance the image contrast before the image to be detected is input into the model. As an example, the comparison table of the detection indicators of the above-mentioned trained defect detection model and other models is shown in Table 1.
[0087] Table 1 Comparison of detection indicators of the above-mentioned trained defect detection models and other models
[0088]
[0089] The unoptimized neural network model in Table 1 is the existing neural network model, and the two separate models are the existing two separate neural network models. During the detection, for the unoptimized neural network model, 60 defective products were input, 41 were detected, and the defect detection rate was 68.33%. 4614 non-defective products were input, 71 were falsely detected, and the false detection rate was 1.54%. For the two separate models, 60 defective products were input, 47 were detected, and the defect detection rate was 78.33%. 4614 non-defective products were input, 108 pieces were falsely detected, the false detection rate was 2.34%, and the total reasoning time was 9.79ms. For the trained defect detection model, 60 defective products were put into use, 56 pieces were detected, and the defect detection rate was 93.33%; 4614 non-defective products were put into use, 93 pieces were falsely detected, the false detection rate was 2.02%, and the total reasoning time was 4.96ms. It can be seen that the above-mentioned trained defect detection model has good performance in false detection rate, detection rate and reasoning time, so it can improve the accuracy and speed of defect detection.
[0090] The shell defect detection method provided by the embodiment of the present invention obtains a shell image to be detected; inputs the shell image to be detected into a trained defect detection model to obtain a shell defect detection result, wherein the defect detection model is obtained by training an initial defect detection model using shell image samples to be detected, and the initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect, and the fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses multiple feature maps of different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using shell image samples to be detected whose bounding box loss is greater than the average bounding box loss; the detection of shell defects can be achieved, and the accuracy and speed of defect detection can be improved.
[0091] A single light source image cannot capture the global features of the detection target. The image features under different light sources are similar, but the unique details of certain positions can be captured from them. If the neural network is trained through images under different light sources, multiple neural network models are obtained and then defect detection is performed separately, which consumes a lot of computing resources and affects the real-time detection speed. The shell defect detection method provided by the embodiment of the present invention abandons the multi-model detection scheme, adds a multi-input lightweight fusion network after the input end of the backbone network, and integrates the unique image features of the auxiliary light source into the main light source image. Guided by the loss function of the backbone network, the attention mechanism can well capture the local information missing from the main light source, and the residual mechanism ensures that the global information is not destroyed. Compared with the existing multi-model detection scheme, a lightweight fusion network is added, which saves computing resources and greatly improves the overall detection speed.
[0092] The shell defect detection method provided by the embodiment of the present invention makes the model converge by re-training the defect samples that are difficult to converge in training, so that the model focuses on the defect features that are difficult to learn, thereby improving the model's recognition ability for such defects. Through selective secondary training, it effectively compensates for the under-learning of certain defect features during training, thereby improving the detection accuracy and enhancing the model performance without increasing the number of training samples.
[0093] The shell defect detection method provided by the embodiment of the present invention can adapt to different lighting conditions through a single model, thereby improving the versatility and applicability of defect detection. Without increasing the number of training samples, it can still significantly improve the model performance, improve the model training efficiency, improve the stability of the training process, and avoid the problem of non-convergence of model training in complex defect situations.
[0094] Based on the above shell defect detection method, the embodiment of the present invention further provides a shell defect detection device, the structural diagram of which is as follows: Figure 3As shown, the housing defect detection device 30 includes a data acquisition module 31 and a detection module 32;
[0095] The data acquisition module 31 is used to acquire the image of the shell to be detected;
[0096] The detection module 32 is used to input the shell image to be detected into a trained defect detection model to obtain a shell defect detection result. The defect detection model is obtained by training an initial defect detection model using shell image samples to be detected. The initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect. The fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses multiple feature maps of different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using shell image samples to be detected whose bounding box loss is greater than the average bounding box loss.
[0097] For other details about how the modules in the shell defect detection device implement the above technical solution, please refer to the description of the shell defect detection method provided in the above invention embodiment, which will not be repeated here.
[0098] Based on the above shell defect detection method, an embodiment of the present invention further provides an electronic device, a structural diagram of which is as follows: Figure 4 As shown, the electronic device 40 includes a processor 41 and a memory 42 coupled to the processor 41. The memory 42 stores a computer program, and when the computer program is executed by the processor 41, the processor 41 executes the steps of the housing defect detection method in the above embodiment.
[0099] For other details about the implementation of the above technical solution by the processor 41 in the above electronic device 40, please refer to the description of the shell defect detection method provided in the above invention embodiment, which will not be repeated here.
[0100] Among them, the processor 41 can also be called a CPU (Central Processing Unit), and the processor 41 may be an integrated circuit chip with signal processing capabilities; the processor 41 can also be a general-purpose processor, DSP (Digital Signal Process), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, among which the general-purpose processor can be a microprocessor or the processor 41 can also be any conventional processor, etc.
[0101] The embodiment of the present invention further provides a computer-readable storage medium, a schematic diagram of which is shown in FIG. Figure 5 As shown, a readable computer program 51 is stored on the storage medium 50; wherein, the computer program 51 can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, magnetic disk or optical disk, ROM (Read-Only Memory), RAM (Random Access Memory) and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.
[0102] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0103] The above embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A shell defect detection method, characterized in that: include: Acquire the shell image to be detected; The shell image to be detected is input into the trained defect detection model to obtain the shell defect detection result. The defect detection model is obtained by training the initial defect detection model using the shell image samples to be detected. The initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect. The fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss.
2. The shell defect detection method according to claim 1, characterized in that: The shell image samples to be detected include main light source image samples and auxiliary light source image samples; accordingly, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, including: The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module to obtain a fused feature map. The fused feature map enters the feature reconstruction module, and the feature reconstruction module introduces an attention module.
3. The shell defect detection method according to claim 2, characterized in that: The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module, including: The feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module according to a fusion formula, and the fusion formula includes Among them, f o (x,y) represents the fused feature map, Represents the pixel with coordinates (x, y) in the jth channel of the ith image sample, c is the total number of channels, the first image sample is the main light source image sample or the auxiliary light source image sample, the second image sample is the auxiliary light source image sample or the main light source image sample, fuse represents the element fusion rule, and the element fusion rule is to take the maximum value, minimum value or average value of each element in the brackets.
4. The shell defect detection method according to claim 1, characterized in that: The shell defect detection method further includes: When the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss, the corresponding total loss is obtained according to the classification loss, the bounding box loss, the confidence loss and the total loss calculation formula. The total loss function calculation formula includes Among them, Loss total_>GIoU is the total loss, 1-GIoU k is the bounding box loss of the kth shell image sample to be detected, E(1-GIoU) is the average bounding box loss, n is the total number of shell image samples to be detected, and loss cls is the classification loss, loss obj is the bounding box loss, loss rect is the confidence loss.
5. The shell defect detection method according to claim 1, characterized in that: The feature extraction infrastructure includes at least one feature extraction module among a Focus module, a Conv module, a C3 module and a spatial pyramid pooling module.
6. The shell defect detection method according to claim 1, characterized in that: The Head includes an FPN module and a PAN module.
7. The shell defect detection method according to claim 1, characterized in that: The Detect includes a convolutional layer.
8. A housing defect detection device, characterized in that: It includes a data acquisition module and a detection module; The data acquisition module is used to acquire the image of the shell to be detected; The detection module is used to input the shell image to be detected into the trained defect detection model to obtain the shell defect detection result. The defect detection model is obtained by training the initial defect detection model using the shell image samples to be detected. The initial defect detection model includes a fusion network, a feature extraction infrastructure, Head and Detect. The fusion network includes a feature extraction module, a feature fusion module and a feature reconstruction module. During the training process, the feature fusion module fuses the feature maps of multiple different convolution kernel sizes generated by the feature extraction module. In the second half of the training, the initial defect detection model is trained using the shell image samples to be detected whose bounding box loss is greater than the average bounding box loss.
9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, wherein: When the processor executes the computer program, the shell defect detection method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the shell defect detection method according to any one of claims 1 to 7 is implemented.