Potato defect detection method for improving image brightness consistency
By improving the EnlightenGAN model, combining segmented attention and SE attention mechanism, the brightness enhancement ability is optimized, and the robustness problem of potato defect detection under different brightness conditions is solved, achieving high-precision and stable defect detection effect.
Patent Information
- Application Number
- CN202510585828.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-15
AI Technical Summary
Existing deep learning models have poor robustness in the detection of potato defects under different image brightness conditions, resulting in unstable detection results.
The improved EnlightenGAN model is adopted, combining the segmented attention mechanism and the SE attention mechanism to optimize the brightness enhancement capabilities, and a data set that is adapted to different lighting conditions is built, and the YOLO11-L object detection framework is trained to improve image brightness consistency and detection stability.
It significantly improves the accuracy and stability of potato defect detection, and can accurately identify defects such as rot, green skin, germination, mechanical damage under complex backgrounds or non-uniform lighting conditions, improving the efficiency of the automated sorting system and the reliability of food safety inspection.
Smart Images
Figure CN120495236A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of agricultural visual inspection and intelligent image processing, and specifically to a method for enhancing brightness consistency and detecting defects in potato images based on an improved EnlightenGAN. Background Art
[0002] Potatoes are the world's third most important food crop and are currently grown in over 150 countries. However, during the planting, harvesting, and processing processes, potatoes are prone to defects such as rot, sprouting, mechanical damage, and green skin. If these defective potatoes are mixed with healthy potatoes during production and processing, they may not only affect the quality of the final product and food safety, but also reduce processing efficiency and, in turn, affect economic benefits. Therefore, the development of efficient and intelligent testing equipment to comprehensively evaluate potato defects is of great significance for ensuring food safety, improving product quality, reducing losses, and promoting the development of intelligent agriculture.
[0003] In recent years, the rapid development of deep learning technology has led to its widespread application in agriculture, demonstrating significant advantages in detecting cosmetic defects in agricultural products. The introduction of deep learning technology can effectively improve the accuracy and automation of potato sorting. However, existing deep learning models suffer from poor robustness when exposed to varying image brightness conditions, which can lead to unstable detection results.
[0004] In actual production environments, detection equipment often encounters complex potato image brightness changes, such as differences in potato varieties and whether there is soil on the surface, which poses a severe challenge to image brightness consistency and affects the stability of the target detection algorithm. Figure 1 As shown in the figure, there are still significant differences in image brightness between different varieties of potatoes under the same shooting conditions.
[0005] Therefore, it is urgent to develop a technology that can improve the brightness consistency of potato images to enhance the stability and adaptability of the detection system. Summary of the Invention
[0006] To address the practical problem of poor image brightness consistency affecting defect detection during intelligent potato inspection and sorting, this paper proposes an integrated method for adaptive potato brightness enhancement and defect detection based on an improved EnlightenGAN. This method is used to effectively improve the brightness consistency of potato images under complex production conditions, enhance the stability and accuracy of defect detection, and improve the brightness consistency of potato images and the accuracy and stability of target detection. The technical solutions of this invention are as follows:
[0007] S1: Use multi-channel potato equipment to collect full-surface images of potatoes, covering various varieties and states (such as with soil or after washing).
[0008] S2: We propose a Potato Image Brightness Rating (PIBR) to analyze the brightness of potato images across their entire surface to quantify the image brightness level. Based on the HSV color space, the PIBR index first accurately extracts the target potato region and eliminates background interference. Within the extracted target region, the mean of the non-zero brightness values in the V channel of the HSV color space is calculated and normalized to serve as the image brightness indicator.
[0009] S3: We deeply optimize the network structure of the image enhancement model EnlightenGAN and propose MS-EnlightenGAN, which integrates the improved self-attention mechanism and SE attention mechanism to significantly improve the model's brightness consistency enhancement capability and applicability.
[0010] S3.1: Self-attention mechanism optimization
[0011] The calculation formula of the original EnlightenGAN's self-attention mechanism is as follows:
[0012] x=1-I
[0013] Where I is the brightness value of the original image after grayscale and normalization.
[0014] However, in practical applications, the original self-attention mechanism still has certain limitations, such as the introduction of background noise and over-enhancement of some areas. To this end, based on the original self-attention mechanism, a segmented attention mechanism is introduced to optimize the brightness enhancement process:
[0015] The captured potato image contains a low-brightness background region (close to pure black) that requires no enhancement. However, the original self-attention mechanism calculates an enhancement weight close to 1 for this region, resulting in excessive background enhancement and unnecessary interference. Testing shows that pixels with 1-I values between 0.96 and 1 are close to the background, so the enhancement weight is set to 0 and no processing is required.
[0016] 2. For the target part of the potato image, pixels with 1-I values between 0.8 and 0.96 have low brightness and defects are difficult to characterize and identify. Therefore, an enhancement weight of 1 is assigned to ensure that the key area is fully enhanced.
[0017] The 1-I value is between 0.5-0.8, maintaining a normal enhancement weight to ensure stable brightness. The 1-I value is between 0-0.5: excessive enhancement may lead to overexposure and weaken the defect characteristics, so an attenuation factor is introduced to suppress excessive enhancement. The optimized piecewise function is as follows:
[0018]
[0019] Among them, f(x) is the final self-attention mechanism enhancement weight.
[0020] The improved self-attention mechanism provides a more rational hierarchical approach to enhancement effects, significantly improving the model's ability to achieve consistent brightness enhancement, ensuring stable and high-quality brightness enhancement of potato images across varying brightness conditions. Furthermore, the model's robustness in diverse environments and applicability to diverse application scenarios have been significantly enhanced, providing a more reliable image input foundation for object detection.
[0021] S3.2: Fusion SE Attention Mechanism
[0022] In the model optimization process of this study, we focused on optimizing the downsampling part of the EnlightenGAN generator network. The downsampling layer of this network consists of multiple convolutional layers, including conv1, conv2, conv3, conv4, and conv5, each of which contains a convolutional block layer and a maximum pooling layer.
[0023] After the max-pooling layers in key layers (conv2, conv3, conv4, and conv5), we introduced the SE module to increase the network's focus on important features, optimize feature representation, and further suppress unnecessary brightness enhancement. This optimization enables the model to more accurately enhance brightness consistency in potato images, improving the stability and adaptability of defect detection.
[0024] S4: To meet the requirements of the image enhancement task, a potato full-surface image enhancement dataset consisting of low-brightness and normal-brightness images was constructed to ensure that the training samples covered different lighting conditions. This dataset was used to train MS-EnlightenGAN.
[0025] At the same time, a target detection dataset is constructed, and YOLO11-L is selected as the target detection framework to evaluate the effectiveness of the method of the present invention and verify the overall performance improvement.
[0026] S5: After training, use MS-EnlightenGAN to enhance the input image, improve the brightness, and highlight the features of the detection target.
[0027] The enhanced images are used as input for analysis and processing by the YOLO11-L object detection network to improve the accuracy and stability of defect detection, while further verifying the effectiveness of the MS-EnlightenGAN method.
[0028] S6: Classify the potatoes according to the test results and output a test report or control the sorting equipment for automated processing.
[0029] The beneficial effects of the present invention are:
[0030] This paper proposes an improved image enhancement model, MS-EnlightenGAN, which combines a segmented attention mechanism with a SE attention module. This model significantly improves the ability and adaptability of enhancing brightness consistency in potato image processing. Compared to existing solutions, this model effectively avoids over-enhancement of background areas while enhancing the brightness expression of defect areas, thereby improving detection accuracy.
[0031] Furthermore, the present invention combines MS-EnlightenGAN with the object detection network YOLO11-L to enhance the recognition of defects such as rot, green skin, sprouting, and mechanical damage in complex backgrounds or under non-uniform lighting conditions, thereby improving the stability and robustness of object detection. This method is suitable for smart agricultural visual inspection scenarios, helping to improve the efficiency of automated sorting systems and enhance the reliability of food safety testing. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a schematic diagram of images of different varieties of potatoes;
[0033] Figure 2 This is a diagram of the MS-EnlightenGAN generator network architecture;
[0034] Figure 3 This is a schematic diagram of the integrated process of image enhancement and defect detection;
[0035] Figure 4 Improve the comparison of image quality indicators under ablation experiments;
[0036] Figure 5 This is a schematic diagram comparing the image enhancement effects of MS-EnlightenGAN;
[0037] Figure 6 This is a comparison chart of the consistency enhancement capabilities of MS-EnlightenGAN;
[0038] Figure 7 This is a schematic diagram of target detection comparison after MS-EnlightenGAN enhancement;
[0039] Figure 8 Schematic diagram of the target detection performance improvement after MS-EnlightenGAN enhancement. DETAILED DESCRIPTION
[0040] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Examples of the implementation are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be understood as limiting the present invention.
[0041] This paper proposes an integrated method for potato adaptive brightness enhancement and defect detection based on an improved EnlightenGAN, aiming to solve the problem of poor image brightness consistency affecting defect detection during potato intelligent detection and sorting. By optimizing the image brightness enhancement strategy, the accuracy and stability of target detection are improved. The overall process of the method of the present invention is as follows: Figure 3 as shown, and includes the following steps:
[0042] S1: Images of different potato varieties Figure 1 As shown in FIG, a multi-channel potato device is used to collect full-surface images of potatoes, covering a variety of varieties and states (such as with soil or after washing).
[0043] S2: The PIBR index is proposed to perform brightness analysis on the full surface image of potato to quantify the image brightness level.
[0044] The PIBR index is based on the HSV color space. It first uses HSV threshold segmentation combined with contour detection technology to accurately locate the main area of the potato and eliminate background interference. Then, within the extracted target area, the mean of the non-zero brightness values of the V channel in the HSV color space is calculated and normalized from [0,255] to [0,1] to serve as the brightness index of the potato image.
[0045] This method not only provides solid data support for subsequent image quality assessment and brightness enhancement strategy formulation, but also lays a theoretical foundation for the quantitative analysis of potato image brightness characteristics and consistency in subsequent related research.
[0046] S3: Deeply optimize the network structure of the image enhancement model EnlightenGAN and propose MS-EnlightenGAN, which integrates the improved self-attention mechanism and SE attention mechanism to significantly improve the model's brightness consistency enhancement capability and applicability. The improved generator network architecture is as follows: Figure 2 shown.
[0047] S3.1: Self-attention mechanism optimization
[0048] The calculation formula of the original EnlightenGAN's self-attention mechanism is as follows:
[0049] x=1-I
[0050] Where I is the brightness value of the original image after grayscale and normalization.
[0051] However, in practical applications, the original self-attention mechanism still has certain limitations, such as the introduction of background noise and over-enhancement of some areas. To this end, based on the original self-attention mechanism, a segmented attention mechanism is introduced to optimize the brightness enhancement process:
[0052] 1. If Figure 1 As shown, the captured potato image contains a low-brightness background region (nearly pure black) that requires no enhancement. However, the original self-attention mechanism calculates an enhancement weight close to 1 for this region, resulting in excessive background enhancement and unnecessary interference. Testing shows that pixels with 1-I values between 0.96 and 1 are close to the background, so the enhancement weight is set to 0 and no processing is required.
[0053] 2. For the main part of the potato, pixels with 1-I values between 0.8 and 0.96 have low brightness, making defects difficult to characterize and identify. Therefore, an enhancement weight of 1 is assigned to ensure that key areas are fully enhanced. For 1-I values between 0.5 and 0.8, a normal enhancement weight is maintained to ensure stable brightness. For 1-I values between 0 and 0.5, excessive enhancement may lead to overexposure, weakening the defect characteristics. Therefore, an attenuation factor is introduced to suppress excessive enhancement. The optimized piecewise function is as follows:
[0054]
[0055] Among them, f(x) is the final enhancement weight.
[0056] S3.2: Fusion SE Attention Mechanism
[0057] In the model optimization process of this study, we focused on optimizing the downsampling part of the EnlightenGAN generator network. The downsampling layer of this network consists of multiple convolutional layers, including conv1, conv2, conv3, conv4, and conv5, each of which contains a convolutional block layer and a maximum pooling layer.
[0058] After the max-pooling layers in key layers (conv2, conv3, conv4, and conv5), we introduced the SE module to increase the network's focus on important features, optimize feature representation, and further suppress unnecessary brightness enhancement. This optimization enables the model to more accurately enhance brightness consistency in potato images, improving the stability and adaptability of defect detection.
[0059] By design, the SE module was not introduced in the conv1 layer. The main reason is that this layer focuses on extracting low-level features. Applying the attention mechanism too early may interfere with the stability of the underlying features and have limited improvement. In addition, the SE module involves global pooling and fully connected calculations. In the conv1 layer, excessively large images will lead to a significant increase in computational overhead. In contrast, the feature maps of conv2 and deeper layers are smaller in size. Introducing the SE module can improve the model's adaptability while avoiding excessive computational burden.
[0060] Therefore, targeted optimization of feature information at different levels in the model can effectively improve the stability and robustness of the image enhancement model.
[0061] S4: Dataset construction
[0062] S4.1: A total of 8492 images were used for training, validation, and testing of the target detection model, of which 6992 were training and validation sets, accounting for 82.35% of the total data volume. To evaluate the robustness of the model under different lighting conditions, a stratified test set containing 1500 images was constructed, accounting for 17.65% of the total data volume. This test set uses a stratified random sampling strategy, uniformly sampling 300 images at every 0.1 index interval within the main distribution range of the PIBR index of 0.2 to 0.7. Considering that there are fewer image samples with PIBR index in the range of 0.1 to 0.2 and 0.7 to 1.0 (less than 5% in total), they are excluded. Finally, a test set with a representative brightness gradient structure is obtained, which can effectively measure the stability of the model under different brightness conditions.
[0063] In particular, the average PIBR index of the images in the training set and the validation set is 0.48. Therefore, in this study, if the PIBR index of an image is ∈ [0, 0.4), it is classified as a low brightness range, and it is believed that images in this range may have a higher misjudgment rate or detection difficulty.
[0064] S4.2: To meet the requirements of the image enhancement task, we divided the images into brightness ranges based on the PIBR index and constructed an image enhancement dataset. A total of 7,403 images were collected, of which 3,276 images had a PIBR index below 0.4, accounting for 44.25% of the total data volume and were classified as the low-brightness image dataset. A total of 4,127 images had a PIBR index greater than or equal to 0.4, accounting for 55.75% and were classified as the normal-brightness image dataset.
[0065] Furthermore, to further validate the model's enhanced performance in scenes with continuously varying brightness, a standardized test set was constructed, independent of the spatiotemporal distribution of the training data. This test set, based on a PIBR index range of 0.2 to 0.7, was sampled in stratified intervals of 0.1, with 20 images per stratum, for a total of 100 images, covering five brightness ranges. This test set provides excellent brightness gradient representation and valuable validation of model generalization.
[0066] S5: Use the image enhancement dataset to train the image enhancement network MS-EnlightenGAN, and use the object detection dataset to train the YOLO11-L object detection model.
[0067] S6: Use the trained MS-EnlightenGAN model to enhance the image, improve the brightness of the image, and highlight the detection target features. The enhanced image is then used as the input of the object detection network YOLO11-L.
[0068] S6.1: Analysis of image enhancement model’s improvement indicators for image enhancement
[0069] Figure 4 The figure shows the image quality improvement of each model under ablation experiments. The data shows that in the luminance range of PIBR of 0.2–0.3, EnlightenGAN achieves an SSIM of 0.75 and a PSNR of 15.19 dB, while SE-EnlightenGAN achieves 0.67 and 13.2 dB, respectively. This indicates that the SE module alone is insufficient in this PIBR range. In contrast, MSA-EnlightenGAN, which utilizes an improved self-attention module, improves SSIM to 0.85 and PSNR to 16.78 dB. Furthermore, MS-EnlightenGAN, which combines the two improvements, achieves an SSIM of 0.87 and a PSNR of 17.48 dB, demonstrating the complementary advantages of the two improvements. The same trend is observed in other luminance ranges, further confirming that MS-EnlightenGAN achieves the best overall performance.
[0070] Figure 5 As shown in Figure 2, the comparison of image enhancement results between MS-EnlightenGAN and EnlightenGAN shows that MS-EnlightenGAN performs well in reducing over-enhancement and background interference problems, and the brightness consistency is also greatly improved.
[0071] Figure 6The model demonstrates enhancement results on a test set of an image enhancement task dataset. Comparative analysis shows that compared to images enhanced by EnlightenGAN, the brightness distribution of images generated by MS-EnlightenGAN is more concentrated, and over-enhancement is effectively reduced. This further validates the model's advantage in improving image brightness consistency, which helps improve the performance and stability of object detection.
[0072] S7: Use the YOLO11-L object detection model to accurately detect defects in the enhanced image, including rot, sprouting, mechanical damage, and greening.
[0073] S7.1: If Figure 7 As shown in the enhanced image, the detection results of the defect area are "rot 0.52"
[0074] "rot 0.70" and "germinated 0.12," where "rot" and "germinated" represent the detected defect types, "rot" for rot defects, and "germinated" for germination defects. The subsequent numbers represent the model's confidence score. Compared to the original image, image enhancement significantly improves the visibility of detailed features such as rot, which not only facilitates accurate defect identification but also increases the model's confidence in the defect type.
[0075] Comparison of detection results shows that enhanced images improve the representation of defect details. This not only enables the model to detect more defects, but also increases the confidence level of detected defects, further validating the optimization effect of image enhancement on object detection performance.
[0076] S7.2: Analysis of improvement of model improvement indicators. The detailed data of the experiment are shown in Table 1. The specific change trends are shown in Figure 8 As shown,
[0077] In the brightness range of PIBR values of 0.2-0.3, images processed by MS-EnlightenGAN demonstrated superior overall performance within the YOLO11-L framework. Its mAP50-95 score increased by 6.45% to 77.6%, slightly lower than EnlightenGAN's 9.47%, but achieving a peak mAP50 score of 87.9%. Furthermore, this method achieved a 3.66% improvement in Precision, surpassing EnlightenGAN's 1.18% gain while maintaining a 14.69% increase in Recall. This demonstrates a more effective improvement in Precision while maintaining a consistent increase in Recall.
[0078] In the brightness range of PIBR values between 0.5 and 0.7, MS-EnlightenGAN continued to demonstrate a stable optimization trend. In particular, in the 0.6-0.7 range, this method achieved a 7.22% improvement in recall, far exceeding EnlightenGAN's 0.38%. mAP50-95 increased by 5.26%, while EnlightenGAN actually decreased by 1.00% under the same conditions. This suggests that EnlightenGAN may introduce overexposure during the enhancement process, affecting the representation of defect features. MS-EnlightenGAN effectively mitigates this issue, making defect areas clearer and improving detection robustness.
[0079] Across all PIBR intervals, under the YOLO11-L framework, EnlightenGAN-processed images achieved average improvements of 2.11%, 2.74%, 1.41%, and 3.23% in Precision, Recall, mAP50, and mAP50-95, respectively. MS-EnlightenGAN achieved 2.52%, 5.28%, 3.41%, and 5.51%, respectively, significantly outperforming EnlightenGAN. This demonstrates that this method not only more effectively enhances image details but also excels in key object detection metrics, providing solid technical support for potato defect detection.
[0080] Table 1 Comparative analysis of target detection performance improvement by different image enhancement models
[0081]
[0082]
[0083]
[0084] S8: Classify the potatoes according to the test results, and output a test report or control the sorting equipment for automated processing.
[0085] The above disclosure is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.
Claims
1. A potato defect detection method for improving image brightness consistency, characterized in that: The following steps are involved: S1: Use multi-channel potato equipment to collect full-surface images of potatoes, covering various varieties and states; S2: The potato image brightness index PIBR is proposed to analyze the brightness of the entire potato surface image and quantify the image brightness level; S3: For the network structure of the image enhancement model EnlightenGAN, we propose MS-EnlightenGAN, which integrates the improved self-attention mechanism and SE attention mechanism; S4: Construct an image enhancement dataset consisting of potato full-surface images of different brightness levels. Use this dataset to train MS-EnlightenGAN, select YOLO11-L as the target detection framework, output defect detection results, and conduct testing.
2. The potato defect detection method for improving image brightness consistency according to claim 1, characterized in that: The potato image brightness index (PIBR) is specifically as follows: the PIBR index is based on the HSV color space. First, the potato target area is extracted and background interference is eliminated. Then, within the extracted target area, the mean of the non-zero brightness values of the V channel in the HSV color space is calculated and normalized to serve as the brightness index of the potato image.
3. The potato defect detection method for improving image brightness consistency according to claim 2, characterized in that: The specific implementation process of the improved self-attention mechanism described in step S3 is as follows The calculation formula of the original EnlightenGAN's self-attention mechanism is as follows: x=1-I Where I is the brightness value of the original image after grayscale and normalization processing; Based on the original self-attention mechanism, a segmented attention mechanism is introduced to optimize the brightness enhancement process: For potato images containing background areas, no enhancement is performed; For the target part of the potato image, the optimized piecewise function is as follows: Among them, f(x) is the final self-attention mechanism enhancement weight.
4. The potato defect detection method for improving image brightness consistency according to claim 3, characterized in that: The specific implementation process of integrating the SE attention mechanism in step S3 is as follows: Optimize the downsampling part of the EnlightenGAN generator network. The downsampling layer consists of five convolutional layers, including, each of which contains a convolutional block layer and a maximum pooling layer; After the maximum pooling layer of the last four convolutional layers, the SE module is introduced to optimize feature expression and suppress unnecessary brightness enhancement effects.
5. The potato defect detection method for improving image brightness consistency according to claim 4, characterized in that: The potato defect detection method further includes classifying potatoes according to the potato defect detection results, and outputting a detection report or controlling sorting equipment to perform automated processing.