YOLOv5-based Cuiguan pear surface defect detection method and system

Abstract

The invention provides a YOLOv5-based Cuiguan pear surface defect detection method and system in the technical field of computer vision and agricultural intellectualization. The method comprises the following steps: S1, acquiring a large number of historical Cuiguan pear images to construct a data set; s2, creating a green-coronal pear defect detection model based on the improved YOLOv5s; s3, training the Cuiguan pear defect detection model through the data set, and performing deployment after training; s4, collecting a real-time Cuiguan pear image to be detected, and inputting the image into the Cuiguan pear defect detection model to obtain a Cuiguan pear defect detection result; and S5, recording a detection log which at least comprises the detection time, the real-time Cuiguan pear image, a Cuiguan pear defect detection result and detection error feedback in real time, and carrying out iterative optimization on the Cuiguan pear defect detection model based on the detection log. The method has the advantage that the accuracy and practicability of surface defect detection of the Cuiguan pears are greatly improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent agriculture, and specifically to a method and system for detecting surface defects in the Cui Guan pear based on YOLOv5. Background Technology

[0002] Fruit surface defect detection is a key technology in the fields of intelligent agriculture and food quality control. For fruits like the Cui Guan pear, whose skin is easily damaged, efficient and accurate defect detection methods can effectively improve production efficiency and product quality. In recent years, deep learning-based target detection algorithms, especially one-stage detection models represented by YOLOv5 (You Only Look Once Version 5), have been widely used in fruit surface defect identification tasks due to their excellent real-time performance and detection accuracy. YOLOv5 can achieve target localization and classification through a single forward propagation, making it very suitable for real-time online detection scenarios. However, existing technologies still have significant shortcomings in the detection of surface defects in Cui Guan pears and urgently need optimization and improvement.

[0003] Currently, most research focuses on common fruits such as apples and citrus fruits, with a lack of research specifically targeting surface defect detection in pears. Existing detection methods typically use general object detection models, such as YOLOv5s and YOLOv5m from the YOLOv5 series. Their structure generally includes a Backbone, Neck (multi-scale feature fusion module), and Head, utilizing a feature pyramid structure to achieve multi-scale object detection. For example, YOLOv5 uses PANet (Path Aggregation Network) for bidirectional feature fusion to balance high-level semantic information with low-level detailed features. Although these methods perform well in defect identification of common fruits, the following key issues remain in practical applications with pears: 1. Dataset scarcity: The surface defects of the Cui Guan pear are diverse, commonly including punctures, rot, branch wounds, abrasions, and black spot disease, among others. However, currently available public datasets lack multi-category image data of Cui Guan pear defects with finely labeled bounding boxes. This insufficiency in training samples makes it difficult for the model to cover the complex defect morphologies in real-world production environments, severely limiting the model's generalization ability and practical application effectiveness.

[0004] 2. Insufficient Model Structure Adaptability: When performing defect detection on Cuiguan pears based on the existing YOLOv5 architecture, the model faces bottlenecks in multi-scale feature fusion and small target recognition. Surface defects in Cuiguan pears often exhibit small-sized features, and the defective areas are similar in color to the normal peel with low contrast. The feature fusion mechanism used in the existing YOLOv5 has limited ability to extract subtle features in complex backgrounds, leading to increased false negatives and missed detections. Furthermore, the attention mechanism and feature weighting strategy built into YOLOv5 are relatively simple, making it difficult to adaptively focus on different categories of defect features, thus affecting the model's robustness in real-world scenarios.

[0005] In summary, the current development of surface defect detection technology for *Pyrus pyrifolia* is limited by the lack of high-quality labeled data and the insufficient adaptability of model structures to specific tasks. The lack of datasets makes model training lack sufficient basis, while the limitations of the existing YOLOv5 model in feature fusion mechanism and attention design further restrict the improvement of detection performance.

[0006] Therefore, how to provide a YOLOv5-based method and system for detecting surface defects in Cuiguan pears, thereby improving the accuracy and practicality of surface defect detection in Cuiguan pears, has become an urgent technical problem to be solved. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a method and system for detecting surface defects of Cuiguan pear based on YOLOv5, so as to improve the accuracy and practicality of surface defect detection of Cuiguan pear.

[0008] In a first aspect, the present invention provides a method for detecting surface defects in the pear tree based on YOLOv5, comprising the following steps: Step S1: Obtain a large number of historical images of Cui Guan pears, and construct a dataset after preprocessing, labeling and sample augmentation of each historical image of Cui Guan pears; Step S2: Create a defect detection model for the Cuiguan pear based on the improved YOLOv5s, and set the loss function of the Cuiguan pear defect detection model; Step S3: Divide the dataset into a training set, a validation set, and a test set. Use the training set, validation set, and test set to train, validate, and test the Cui Guan pear defect detection model, respectively. Deploy the Cui Guan pear defect detection model that passes the test. Step S4: Collect real-time images of the pears to be detected, input the real-time images of the pears into the deployed pear defect detection model, obtain the pear defect detection results, and display the pear defect detection results. Step S5: Record in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback, and iteratively optimize the defect detection model of the Cui Guan pear based on the detection log.

[0009] Furthermore, step S1 specifically includes: A large number of historical images of Cui Guan pears were acquired, and each of the historical Cui Guan pear images was subjected to preprocessing including image enhancement, size normalization, background removal, and format unification. Each of the historical Cuiguan pear images is smoothed by filtering and then converted to HSV format. A mask image of the region of interest is obtained based on the HSV color range of Cuiguan pear. Each mask image is then bitwise ANDed with the corresponding historical Cuiguan pear image to segment the initial defect region. The historical Cuiguan pear image carrying the initial defect region is then converted to grayscale and dilated. The dilated historical Cuiguan pear image is segmented using a global threshold segmentation method to obtain the precise defect region. Each precise defect region is labeled with a defect category; the defect category includes at least normal fruit, branch injury, mechanical damage, and rot. Based on a preset area threshold, small target defects are selected from each of the precise defect regions. After random scaling, random rotation, and random brightness adjustment of each small target, they are randomly copied onto each of the historical Cui Guan pear images to perform sample augmentation. A dataset is constructed based on the historical Cui Guan pear images after sample augmentation.

[0010] Furthermore, in step S2, the defect detection model for the 'Cuiguan' pear is constructed based on the backbone network, the neck network, and the detection head; The backbone network is used to extract image features at different levels from the input image of the Cui Guan pear; the neck network is used to fuse the image features to obtain fused features; the detection head is used to infer the fused features and output the defect detection results of the Cui Guan pear carrying the defect region and defect category. The backbone network is based on the sequentially connected first downsampling layer, second downsampling layer, first C3True module, third downsampling layer, second C3True module, fourth downsampling layer, third C3True module, fifth downsampling layer, fourth C3True module, ECA module and SPPF module; The first downsampling layer uses a 6×6 convolution kernel with a stride of 2; the second, third, fourth, and fifth downsampling layers all use a 3×3 convolution kernel with a stride of 2; the first, second, third, and fourth C3True modules are all high-precision C3 modules. The neck network is constructed based on a first feature integration layer, a first UpSample module, a first concat module, a first C3False module, a second feature integration layer, a second UpSample module, a second concat module, a second C3False module, a first CA module, a sixth downsampling layer, a third concat module, a third C3False module, a second CA module, a seventh downsampling layer, a fourth concat module, a fourth C3False module, and a third CA module connected in sequence. The input of the first feature integration layer is connected to the output of the SPPF module; the input of the first concat module is connected to the output of the third C3True module; the input of the second concat module is connected to the output of the second C3True module; the input of the third concat module is connected to the output of the second UpSample module; and the input of the fourth concat module is connected to the output of the first feature integration layer. The first and second feature integration layers both use 1×1 convolution kernels with a stride of 1; the sixth and seventh downsampling layers both use 3×3 convolution kernels with a stride of 2; the first, second, third, and fourth C3False modules are all low-precision C3 modules. The detection head is constructed based on a first predictive convolutional layer, a second predictive convolutional layer, and a third predictive convolutional layer; the input of the first predictive convolutional layer is connected to the output of the first CA module; the input of the second predictive convolutional layer is connected to the output of the second CA module; and the input of the third predictive convolutional layer is connected to the output of the third CA module.

[0011] Furthermore, step S3 specifically includes: The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 based on the hold-out method. The defect detection model of the Cui Guan pear is trained using the training set until the loss value of the loss function is less than the preset loss threshold, the validation loss no longer improves, or the maximum epoch is reached. The trained Cui Guan pear defect detection model is validated using the validation set. If the validation fails, the training set is expanded and training continues. If the validation passes, the Cui Guan pear defect detection model is tested using the test set. If the test fails, the training set is expanded and training continues. If the test passes, the Cui Guan pear defect detection model that has passed the test is deployed.

[0012] Furthermore, step S5 specifically includes: The detection log is recorded in real time, including at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback. The detection log is stored and backed up in a distributed manner. Based on a preset iteration cycle, the corresponding detection logs are selected to construct an incremental dataset. The deployed Cuiguan pear defect detection model is trained based on the incremental dataset, and then the deployed Cuiguan pear defect detection model is continuously iterated and optimized.

[0013] Secondly, the present invention provides a YOLOv5-based surface defect detection system for the Cuiguan pear, comprising the following modules: The dataset construction module is used to acquire a large number of historical images of Cui Guan pears, and to construct a dataset after preprocessing, labeling and expanding the samples of each historical Cui Guan pear image. The module for creating a defect detection model for Cuiguan pear is used to create a defect detection model for Cuiguan pear based on the improved YOLOv5s and to set the loss function of the defect detection model for Cuiguan pear. The Cui Guan pear defect detection model training module is used to divide the dataset into a training set, a validation set, and a test set, and to train, validate, and test the Cui Guan pear defect detection model using the training set, validation set, and test set respectively, and to deploy the Cui Guan pear defect detection model that passes the test. The Cuiguan pear defect detection module is used to acquire real-time images of Cuiguan pears to be detected, input the real-time images of Cuiguan pears into the deployed Cuiguan pear defect detection model, obtain the Cuiguan pear defect detection results, and display the Cuiguan pear defect detection results; The model iteration and optimization module is used to record in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback, and to iteratively optimize the defect detection model of the Cui Guan pear based on the detection log.

[0014] Furthermore, the dataset construction module is specifically used for: A large number of historical images of Cui Guan pears were acquired, and each of the historical Cui Guan pear images was subjected to preprocessing including image enhancement, size normalization, background removal, and format unification. Each of the historical Cuiguan pear images is smoothed by filtering and then converted to HSV format. A mask image of the region of interest is obtained based on the HSV color range of Cuiguan pear. Each mask image is then bitwise ANDed with the corresponding historical Cuiguan pear image to segment the initial defect region. The historical Cuiguan pear image carrying the initial defect region is then converted to grayscale and dilated. The dilated historical Cuiguan pear image is segmented using a global threshold segmentation method to obtain the precise defect region. Each precise defect region is labeled with a defect category; the defect category includes at least normal fruit, branch injury, mechanical damage, and rot. Based on a preset area threshold, small target defects are selected from each of the precise defect regions. After random scaling, random rotation, and random brightness adjustment of each small target, they are randomly copied onto each of the historical Cui Guan pear images to perform sample augmentation. A dataset is constructed based on the historical Cui Guan pear images after sample augmentation.

[0015] Furthermore, in the defect detection model creation module for the Cuiguan pear, the defect detection model for the Cuiguan pear is constructed based on the backbone network, the neck network, and the detection head; The backbone network is used to extract image features at different levels from the input image of the Cui Guan pear; the neck network is used to fuse multiple image features to obtain fused features; the detection head is used to infer the fused features and output the defect detection results of the Cui Guan pear carrying the defect region and defect category. The backbone network is based on the sequentially connected first downsampling layer, second downsampling layer, first C3True module, third downsampling layer, second C3True module, fourth downsampling layer, third C3True module, fifth downsampling layer, fourth C3True module, ECA module and SPPF module; The first downsampling layer uses a 6×6 convolution kernel with a stride of 2; the second, third, fourth, and fifth downsampling layers all use a 3×3 convolution kernel with a stride of 2; the first, second, third, and fourth C3True modules are all high-precision C3 modules. The neck network is constructed based on a first feature integration layer, a first UpSample module, a first concat module, a first C3False module, a second feature integration layer, a second UpSample module, a second concat module, a second C3False module, a first CA module, a sixth downsampling layer, a third concat module, a third C3False module, a second CA module, a seventh downsampling layer, a fourth concat module, a fourth C3False module, and a third CA module connected in sequence. The input of the first feature integration layer is connected to the output of the SPPF module; the input of the first concat module is connected to the output of the third C3True module; the input of the second concat module is connected to the output of the second C3True module; the input of the third concat module is connected to the output of the second UpSample module; and the input of the fourth concat module is connected to the output of the first feature integration layer. The first and second feature integration layers both use 1×1 convolution kernels with a stride of 1; the sixth and seventh downsampling layers both use 3×3 convolution kernels with a stride of 2; the first, second, third, and fourth C3False modules are all low-precision C3 modules. The detection head is constructed based on a first predictive convolutional layer, a second predictive convolutional layer, and a third predictive convolutional layer; the input of the first predictive convolutional layer is connected to the output of the first CA module; the input of the second predictive convolutional layer is connected to the output of the second CA module; and the input of the third predictive convolutional layer is connected to the output of the third CA module.

[0016] Furthermore, the defect detection model training module for the Cuiguan pear is specifically used for: The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 based on the hold-out method. The defect detection model of the Cui Guan pear is trained using the training set until the loss value of the loss function is less than the preset loss threshold, the validation loss no longer improves, or the maximum epoch is reached. The trained Cui Guan pear defect detection model is validated using the validation set. If the validation fails, the training set is expanded and training continues. If the validation passes, the Cui Guan pear defect detection model is tested using the test set. If the test fails, the training set is expanded and training continues. If the test passes, the Cui Guan pear defect detection model that has passed the test is deployed.

[0017] Furthermore, the model iterative optimization module is specifically used for: The detection log is recorded in real time, including at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback. The detection log is stored and backed up in a distributed manner. Based on a preset iteration cycle, the corresponding detection logs are selected to construct an incremental dataset. The deployed Cuiguan pear defect detection model is trained based on the incremental dataset, and then the deployed Cuiguan pear defect detection model is continuously iterated and optimized.

[0018] The advantages of this invention are: 1. A dataset was constructed by acquiring a large number of historical images of Cuiguan pears, preprocessing, labeling, and expanding the samples of each image. Then, a defect detection model for Cuiguan pears was created based on an improved YOLOv5s, and the loss function of the model was set. The dataset was divided into training, validation, and test sets. The defect detection model was trained, validated, and tested using these sets respectively. The tested model was then deployed. Next, real-time images of Cuiguan pears were acquired and input into the deployed model to obtain and display the defect detection results. A detection log was recorded in real-time, including at least the detection time, the real-time image of the pear, the defect detection result, and the detection error feedback. The defect detection model was iterated based on the detection log. Optimization was achieved through a two-pronged approach: constructing a high-quality dataset and customizing the detection model. This significantly improved the accuracy and practicality of surface defect detection in the Cuiguan pear. Specifically, a refined annotation method based on HSV color space and threshold segmentation was adopted, combined with specialized data augmentation for small target defects, ensuring the quality and diversity of training data from the source. Key structural improvements were made to the YOLOv5 model, introducing a high-precision C3True module into the backbone network to enhance feature extraction capabilities, and integrating multiple attention mechanisms such as ECA and CA into the neck network. This enabled the model to adaptively focus on low-contrast, small-sized defect features, effectively improving the accuracy of recognition in complex backgrounds. By establishing a closed-loop optimization system that includes log recording and incremental learning, the model can continuously iterate and evolve in practical applications, ultimately greatly improving the accuracy and practicality of surface defect detection in Cuiguan pears.

[0019] 2. By setting up a data preprocessing workflow, including image enhancement, size standardization, background removal, and format unification, as well as accurate extraction of defect regions based on HSV color space and threshold segmentation, this preprocessing not only improves image quality but also effectively increases the diversity and representativeness of the dataset through sample expansion for small target defects (such as random scaling, rotation, and brightness adjustment). This helps alleviate the data imbalance problem in deep learning model training, improves the model's ability to detect rare or small defects, thereby enhancing the robustness and accuracy of the overall system and laying a reliable data foundation for practical applications.

[0020] 3. Targeted optimizations were made to the YOLOv5s model, including the introduction of ECA and C3True modules into the backbone network to enhance feature extraction capabilities, and the integration of a CA module into the neck network to improve feature fusion. This structural design prioritizes key areas on the surface of the pear tree through multi-scale feature fusion and attention mechanisms, thereby significantly improving the accuracy of defect localization and classification (such as distinguishing between branch wounds, mechanical damage, and rot) while maintaining real-time detection speed. In addition, specific convolutional kernels and downsampling layers were used to optimize computational efficiency, enabling the model to run efficiently on resource-constrained devices and demonstrating good industrial applicability.

[0021] 4. By dividing the dataset into training, validation, and test sets using the hold-out method and conducting multiple rounds of training and testing, the model's generalization ability on unknown data is ensured. This rigorous validation process can promptly identify overfitting or underfitting issues and iteratively adjust them through data augmentation, thereby guaranteeing the stability and reliability of the deployed model. This not only reduces the risk of model failure in practical applications but also extends the model's lifespan, meeting the high standards required for automated inspection systems in industrial production.

[0022] 5. The entire process from real-time image acquisition to detection result display is automated, and continuous optimization is achieved by recording detection logs (including time, images, results, and feedback). This iterative mechanism allows the system to continuously fine-tune the model based on actual usage data, adapt to environmental changes (such as lighting and seasonal factors) or the emergence of new defect types, demonstrating the system's adaptability and scalability. It can maintain high detection performance for a long time, reduce manual maintenance costs, and improve overall economic benefits.

[0023] 6. It supports the labeling and detection of multiple defect categories, such as branch wounds, mechanical damage, and rot, covering common surface problems of Cui Guan pears. This multi-category detection capability enables the system to handle diverse quality inspection needs in one stop, avoiding the complexity of using multiple dedicated models. At the same time, combined with precise defect area segmentation, it helps to achieve graded screening or traceability management, providing comprehensive support for agricultural product quality control and supply chain optimization.

[0024] 7. The process from data construction to model deployment is highly automated, reducing the need for manual intervention. For example, through standardized preprocessing and one-click model training, even non-expert users can operate it easily. This design lowers the technical threshold and operating costs, and is easy to integrate into existing agricultural IoT or smart sorting equipment, promoting the intelligent upgrading of the Cui Guan pear industry and conforming to the development trend of modern precision agriculture.

[0025] 8. Integrated data preprocessing and sample augmentation methods (such as random copying for small target defects and HSV mask segmentation) effectively improved data quality. Combined with the improved YOLOv5s model architecture (introducing ECA / CA attention modules and optimizing convolutional layers), it achieved high-precision and high-efficiency detection of multiple types of defects on the surface of Cuiguan pears. Its systematic training and verification process and real-time log recording mechanism further ensured the stability of the model and its adaptive iterative optimization capabilities, ultimately forming an end-to-end automated detection closed loop. This significantly improved the accuracy and sustainability of agricultural quality control while reducing labor costs. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] Figure 1 This is a flowchart of a surface defect detection method for Prunus mume based on YOLOv5 according to the present invention.

[0028] Figure 2 This is a schematic diagram of the structure of a YOLOv5-based surface defect detection system for pears according to the present invention.

[0029] Figure 3 This is a network structure diagram of the defect detection model for the Cui Guan pear of this invention.

[0030] Figure 4 This is a network structure diagram of the ECA module of the present invention.

[0031] Figure 5 This is a network structure diagram of the CA module of this invention.

[0032] Figure 6 This is a schematic diagram of the threshold pear image segmentation process based on a mask according to the present invention.

[0033] Figure 7 This is a schematic diagram of the sample expansion of the present invention. Detailed Implementation

[0034] The overall approach of the technical solution in this application is as follows: A refined annotation method based on HSV color space and threshold segmentation is adopted, combined with specialized data augmentation for small target defects, ensuring the quality and diversity of training data from the source; key structural improvements are made to the YOLOv5 model, introducing a high-precision C3True module into the backbone network to enhance feature extraction capabilities, and integrating multiple attention mechanisms such as ECA and CA into the neck network, enabling the model to adaptively focus on low-contrast, small-sized defect features, effectively improving the accuracy of identification in complex backgrounds; by establishing a closed-loop optimization system including log recording and incremental learning, the model can continuously iterate and evolve in practical applications, thereby improving the accuracy and practicality of surface defect detection in the Cuiguan pear.

[0035] Please refer to Figures 1 to 7 As shown, a preferred embodiment of the YOLOv5-based surface defect detection method for *Pyrus pyrifolia* pears according to the present invention includes the following steps: Step S1: Obtain a large number of historical images of Cui Guan pears, and construct a dataset after preprocessing, labeling and sample augmentation of each historical image of Cui Guan pears; Step S2: Create a defect detection model for the Cuiguan pear based on the improved YOLOv5s and set the loss function of the Cuiguan pear defect detection model. There are multiple versions of YOLOv5 (s,m,l,x), which have similar structures. The main difference lies in the two dimensions of depth (number of network layers) and width (number of channels). YOLOv5s is the version with the shallowest depth and the fewest channels, so it is the fastest.

[0036] Step S3: Divide the dataset into a training set, a validation set, and a test set. Use the training set, validation set, and test set to train, validate, and test the Cui Guan pear defect detection model, respectively. Deploy the Cui Guan pear defect detection model that passes the test. Step S4: Collect real-time images of the pears to be detected, input the real-time images of the pears into the deployed pear defect detection model, obtain the pear defect detection results, and display the pear defect detection results. Step S5: Record in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback, and iteratively optimize the defect detection model of the Cui Guan pear based on the detection log.

[0037] Step S1 specifically involves: A large number of historical images of Cui Guan pears were acquired, and each of the historical Cui Guan pear images was subjected to preprocessing including image enhancement, size normalization, background removal, and format unification. Each historical Cuiguan pear image is smoothed by filtering and then converted to HSV format. A mask image of the region of interest is obtained based on the HSV (hue, saturation, brightness) color range of the Cuiguan pear. Each mask image is then bitwise ANDed with the corresponding historical Cuiguan pear image to segment the initial defect region. The historical Cuiguan pear image carrying the initial defect region is then converted to grayscale and dilated. The dilated historical Cuiguan pear image is segmented using a global thresholding method (binarization is performed, and finally, the resulting binary image is masked again with the original image) to obtain precise defect regions. Labelme data annotation tool is used to label each precise defect region with its defect category. The annotation results are saved in JSON format. The defect categories include at least normal fruit, branch wounds, mechanical damage, and rot. The HSV color space is well-suited to the human visual system, and the ranges of its various color components are clearly defined. Based on the value ranges of each color in the HSV color space, the ranges of each component of the "Cuiguan pear" color are determined. The "Cuiguan pear" color mainly includes yellow and green, with yellow having a smaller value range and green having a larger value range. Therefore, the H, S, and V values ​​of orange are set as the lower limit of the mask color range, while the three green components are set as the upper limit of the mask color range.

[0038] Based on a preset area threshold, small target defects are selected from each of the precise defect regions. After random scaling, random rotation, and random brightness adjustment of each small target, they are randomly copied onto each of the historical green crown pear images (the newly pasted target maintains a distance of at least 10 pixels from the original object to avoid overlap) to perform sample augmentation. A dataset is constructed based on the historical green crown pear images after sample augmentation.

[0039] Although the dataset contains various defect categories of the Cui Guan pear, the class imbalance is addressed from both the data and algorithmic levels. At the data level, images of the minority defect categories are transformed to create more diverse new samples (sample augmentation). This involves applying basic data augmentation operations such as rotation, flipping, scaling, cropping, brightness adjustment, and contrast adjustment to the minority defect category images. Specifically, a GAN (Generative Adversarial Network) is used to generate highly realistic and diverse new images for the minority defect category images, thereby adjusting the distribution of the training data to be closer to balance. At the algorithmic level, the loss function is modified to make the model focus more on the minority class. Specifically, an adjustment factor is used to reduce the loss contribution of "easy-to-classify" samples (usually the majority class), allowing the model to focus more on "hard-to-classify" samples (usually the minority class). The formula is: FL(p_t)=-α_t(1-p_t)^γlog(p_t); Where p_t represents the probability that the model predicts the target class; (1-p_t)^γ represents the modulation factor. When p_t is large (i.e. easy to classify), this modulation factor is close to 0, and the overall loss becomes smaller. That is, by focusing on the parameter γ, the weight of easy-to-classify samples is reduced, so that the model focuses more on difficult-to-classify samples during training. When it is increased (the value of γ is increased), the model will be more "focused" on those "difficult samples" that it has not yet learned and whose classification is ambiguous; α_t represents the class balancing weight.

[0040] In step S2, the defect detection model for the Cuiguan pear is constructed based on the backbone network, the neck network, and the head. The backbone network is used to extract image features at different levels from the input image of the Cui Guan pear; the neck network is used to fuse the image features to obtain fused features; the detection head is used to infer the fused features and output the defect detection results of the Cui Guan pear carrying the defect region and defect category. The defect detection results of the Cuiguan pear also carry confidence and category probability; the role of Neck is to build a powerful feature pyramid that integrates deep high-level semantic features and shallow fine-grained localization features.

[0041] The backbone network is based on a series of sequentially connected layers: a first downsampling layer, a second downsampling layer, a first C3True module (CSP Bottleneck with 3 convolutions, inspired by CSPNet, designed to enhance the learning ability of CNNs by splitting feature maps while reducing computation), a third downsampling layer, a second C3True module, a fourth downsampling layer, a third C3True module, a fifth downsampling layer, a fourth C3True module, an ECA module, and an SPPF module. An ECA module is added before the SPFF module to perform a lightweight channel-level refinement before high-level semantic features enter the Neck. The ECA module has minimal computational overhead, adds almost no latency, but effectively improves feature quality, providing a better starting point for the subsequent Neck.

[0042] C3True enhances the model's representational power by keeping the number of channels constant (rather than reducing dimensionality) in the Bottleneck branch; when the primary goal is to achieve the highest accuracy and the model's speed and size are less of a concern, using a structure that includes C3True is a reasonable choice.

[0043] The SPPF module, or Spatial Pyramid Pooling module, uses multiple max-pooling kernels of different sizes (such as 5x5, 9x9, 13x13) to pool feature maps in parallel, and then concatenates the results. This allows the network to be unrestricted by the input size and greatly increases the receptive field of the backbone features, which helps to detect objects of different scales. SPPF is an accelerated version of SPP, achieving the same effect as SPP by serializing multiple 5x5 pooling layers, but with faster computation.

[0044] The first downsampling layer uses a 6×6 convolution kernel with a stride of 2; the second, third, fourth, and fifth downsampling layers all use a 3×3 convolution kernel with a stride of 2; the first, second, third, and fourth C3True modules are all high-precision C3 modules. The neck network is constructed based on a first feature integration layer, a first UpSample module, a first concat module, a first C3False module, a second feature integration layer, a second UpSample module, a second concat module, a second C3False module, a first CA module, a sixth downsampling layer, a third concat module, a third C3False module, a second CA module, a seventh downsampling layer, a fourth concat module, a fourth C3False module, and a third CA module connected in sequence. The input of the first feature integration layer is connected to the output of the SPPF module; the input of the first concat module is connected to the output of the third C3True module; the input of the second concat module is connected to the output of the second C3True module; the input of the third concat module is connected to the output of the second UpSample module; and the input of the fourth concat module is connected to the output of the first feature integration layer. The neck network uses a two-layer routing mechanism and employs sparse sampling, which preserves more fine-grained details and demonstrates significant superiority in small target detection.

[0045] By combining two different attention mechanisms, ECA (Efficient Channel Attention) and CA (Coordinate Attention), in YOLOv5s, attention is introduced at different macro-level stages of feature extraction. This allows attention mechanisms with different structures and functions to complement each other, enhancing feature representation from multiple dimensions. ECA performs global, lightweight channel preprocessing at the backbone end, improving overall feature quality; CA performs scale-based, fine-grained channel-space joint optimization at the neck output end, particularly improving the localization capability of multi-scale targets. These two attention mechanisms, one preceding and one following, one lightweight and one heavy, one channel and one space, form a complementary and synergistic effect.

[0046] The first and second feature integration layers both use 1×1 convolution kernels with a stride of 1; the sixth and seventh downsampling layers both use 3×3 convolution kernels with a stride of 2; the first, second, third, and fourth C3False modules are all low-precision C3 modules. The core processes of the neck network are: 1. Top-down approach (FPN): The deepest features of the backbone (e.g., 20x20x512) are upsampled and then concatenated with features of the corresponding scale in the backbone (e.g., 40x40x256). This process is repeated to pass high-level semantic information to low-level features.

[0047] 2. Bottom-up (PAN): This is the key to PANet. After completing the top-down fusion, the fused low-level features are downsampled and then concatenated with the fused features from the previous layer. This process passes the precise localization information from the lower layers to the higher-level features.

[0048] Through this bidirectional fusion, PANet enables feature maps at each scale to simultaneously contain rich semantic information and precise location information.

[0049] This improvement is based on YOLOv5s and incorporates the CA attention mechanism. A CA-Net is added to each output of the Neck to jointly refine feature maps of different scales (small, medium, and large targets) using both channel and spatial dimensions. CA attention can capture cross-channel information and directional positional information, which is crucial for target localization, especially helping the model to better outline targets.

[0050] 1. ECA attention mechanism ECA attention mechanism is a channel-based attention mechanism. Its advantage is that it is very lightweight and adds almost no latency, but its disadvantage is that it only focuses on channel relationships.

[0051] ECA is an improved version of the Squeeze-and-Excitation (SE) attention mechanism. The SE attention mechanism "compresses" features through global average pooling, then learns the importance (weights) of each channel through two fully connected layers, and finally "encourages" the original features (channel weighting). The ECA attention mechanism, however, uses one-dimensional convolutions instead of the two fully connected layers in SE, avoiding dimensionality reduction and reconstruction, further reducing the number of parameters and computation while maintaining performance. The core idea of ​​ECA is to capture cross-channel interaction information while avoiding dimensionality reduction. It argues that the dimensionality reduction operation of the fully connected layers in SE attention has side effects on channel attention prediction, therefore using a more efficient, non-dimensionality-reducing method—one-dimensional convolutions—to achieve this.

[0052] It consists of five steps: Step 1: Input feature map: Assume the input feature map size is H×W×C (height×width×number of channels).

[0053] Step 2: Global Average Pooling The spatial information (H×W) of each channel is compressed and converted into a specific numerical value. This step is the same as in the SE module.

[0054] Output: A 1×1×C feature vector, where each value represents global information for one channel.

[0055] Step 3: One-dimensional convolution: This is the innovation and core of the ECA module. Instead of using the two fully connected layers in the SE module, it uses a one-dimensional convolution with a kernel size of k.

[0056] Function: To capture the interaction information between adjacent channels. The kernel size k determines how many of its left and right neighboring channels are considered when calculating attention for each channel. The size of k can be adaptively calculated using a formula.

[0057] Output: A new 1×1×C weight vector after cross-channel information interaction. Note: The dimensions are not changed here; the number of channels C remains unchanged.

[0058] Step 4: Sigmoid activation function: The weight vector output by the one-dimensional convolution is normalized to the range (0,1) to obtain the final attention weight for each channel.

[0059] Step 5: Weighted Allocation for Each Channel The obtained 1×1×C channel attention weights are multiplied channel by channel with the input H×W×C feature map.

[0060] Effect: Important channels are enhanced, while unimportant channels are suppressed.

[0061] Another ingenious design feature of the ECA module is that the kernel size k is not manually set, but adaptively determined based on the number of channels C. There is a mapping relationship between the two: ; in, This indicates taking the nearest odd number; γ and b are hyperparameters, typically set to 2 and 1 respectively. This formula ensures that the coverage of interactions is adaptive for networks of different depths (different number of channels C). Simply put, the more channels there are, the larger the range of interactions (k) that needs to be considered.

[0062] 2. CA Attention Mechanism: CA attention mechanism is a hybrid channel + spatial attention mechanism. Its advantage is that it can simultaneously capture channel relationships and long-range dependencies (with directionality), which is particularly beneficial for target localization. It offers excellent performance with manageable overhead.

[0063] The principle of CA attention mechanism is to decompose channel attention into two parallel 1D global pooling operations along the X and Y directions respectively, thereby capturing the relationship between channels and embedding positional information into the attention graph.

[0064] It consists of three steps: Step 1: Embedding coordinate information: Instead of using global average pooling, CA uses bidirectional pooling to encode global spatial information into a pair of orientation-aware feature maps.

[0065] Input: Feature map X, with dimensions H×W×C.

[0066] X-axis pooling: Uses a pooling kernel of size (H,1) to pool each column of each channel.

[0067] Output: A 1×W×C feature vector Z_h. It captures the global receptive field in the width direction while preserving the positional information in the horizontal direction.

[0068] Y-axis pooling: Uses a pooling kernel of size (1, W) to pool each row of each channel.

[0069] Output: A feature vector Z_w of size H×1×C. It captures the global receptive field in the height direction while preserving the positional information in the vertical direction.

[0070] Step 2: Coordinate Information Fusion and Interaction This is the most ingenious part of CA, which integrates and interacts features from two directions.

[0071] Concatenation and dimensionality reduction: Concatenate Z_h and Z_w, and then perform dimensionality reduction and feature fusion through a shared 1x1 convolution F1 to reduce the number of channels to C / r (r is the reduction rate).

[0072] Nonlinear activation: Use nonlinear activation functions (such as Swish / Sigmoid) to process the fused features.

[0073] Feature splitting: The activated features are split into two independent tensors along the spatial dimension: F_h (size 1×W×C / r) and F_w (size H×1×C / r).

[0074] Convolution and restoration dimensions: Apply 1x1 convolutions F2 and F3 to F_h and F_w respectively to restore the number of channels to the original C, and obtain the intermediate weights G_h and G_w.

[0075] Step 3: Weighted Output The obtained direction-aware weights are applied to the original input feature map.

[0076] Generate attention weights: Apply the Sigmoid function to G_h and G_w respectively to obtain the final attention weights g_h(1×W×C) and g_w(H×1×C).

[0077] Element-wise multiplication: Multiply the weights g_h and g_w element-wise with the original input feature map X.

[0078] g_h weights features in the width direction, emphasizing important column regions.

[0079] g_w weights features in the height direction, emphasizing important row regions.

[0080] Output: The final refined feature map Y is obtained, with dimensions still being H×W×C.

[0081] The detection head is constructed based on a first predictive convolutional layer, a second predictive convolutional layer, and a third predictive convolutional layer; the input of the first predictive convolutional layer is connected to the output of the first CA module; the input of the second predictive convolutional layer is connected to the output of the second CA module; and the input of the third predictive convolutional layer is connected to the output of the third CA module.

[0082] The detection head is the final detection part, receiving feature maps of three different scales from the neck network: large-scale feature map (e.g., 80x80): small receptive field, used to detect small objects; medium-scale feature map (e.g., 40x40): used to detect medium-sized objects; small-scale feature map (e.g., 20x20): large receptive field, used to detect large objects.

[0083] Output for each scale: For each cell of each feature map, the detection head predicts N anchor boxes. For each anchor box, it outputs a vector: [x,y,w,h,obj_score,class1_score,class2_score,...] Where (x,y,w,h) represents the center coordinates and width and height of the bounding box (offsets relative to the cell and anchor point); obj_score represents the confidence score that the bounding box contains an object; class_score represents the probability that the bounding box belongs to each defect category.

[0084] The front part of the detector head has three prediction convolutional layers, which are equivalent to three independent, scale-specific front adapters for the detector head. Their functions are: independent processing: processing the fused feature maps of three different scales in parallel and independently; feature specialization: converting the general fused features into features specifically for bounding box prediction and classification; and channel adjustment: adjusting the number of channels of the feature map to the dimension desired by the detector head.

[0085] They are the key bridges connecting the Feature Pyramid Network (PANet) and the multi-scale prediction head (Detect). In the Neck part, the network outputs feature maps at three different scales (called P3, P4, and P5) through upsampling and downsampling, which are used to detect small, medium, and large objects.

[0086] P3: From a shallower layer, with the largest feature map size (e.g., 80x80), responsible for detecting small objects. P4: From a middle layer, with a medium-sized feature map size (e.g., 40x40), responsible for detecting medium-sized objects. P5: From a deeper layer, with the smallest feature map size (e.g., 20x20), responsible for detecting large objects.

[0087] After feature fusion, the three fused feature maps (P3, P4, P5) cannot be directly fed into the Detect head for prediction. They need to undergo a "refinement" step, which is done by these three independent prediction convolutional layers. Each prediction convolutional layer independently processes a feature map at one scale. Their role is to perform final feature transformation and channel adjustment, specializing the general fused features output by the feature pyramid network into features suitable for the object detection task. The three prediction convolutional layers work in parallel: The first Conv: receives P3 and outputs 80x80x256. This is the final feature map prepared for the small object detection head.

[0088] The second Conv: receives P4 and outputs a 40x40x512 pixel graph. This is the final feature map prepared for the mid-target detection head.

[0089] The third Conv: receives P5 and outputs 20x20x1024. This is the final feature map prepared for a large target detection head.

[0090] Finally, the Detect layer receives these three "refined" feature maps [24,25,26] (i.e., the outputs of the three Conv layers) and performs object detection directly on them (predicted bounding box coordinates, confidence, and class).

[0091] Step S3 specifically involves: The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 based on the hold-out method. The defect detection model of the Cui Guan pear is trained using the training set until the loss value of the loss function is less than the preset loss threshold, the validation loss no longer improves, or the maximum epoch is reached. The trained Cui Guan pear defect detection model is validated using the validation set. If the validation fails, the training set is expanded and training continues. If the validation passes, the Cui Guan pear defect detection model is tested using the test set. If the test fails, the training set is expanded and training continues. If the test passes, the Cui Guan pear defect detection model that has passed the test is deployed.

[0092] Step S5 specifically involves: The detection log is recorded in real time, including at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback. The detection log is stored and backed up in a distributed manner. Based on a preset iteration cycle, the corresponding detection logs are selected to construct an incremental dataset. The deployed Cuiguan pear defect detection model is trained based on the incremental dataset, and then the deployed Cuiguan pear defect detection model is continuously iterated and optimized.

[0093] A preferred embodiment of the YOLOv5-based surface defect detection system for the Cuiguan pear of the present invention includes the following modules: The dataset construction module is used to acquire a large number of historical images of Cui Guan pears, and to construct a dataset after preprocessing, labeling and expanding the samples of each historical Cui Guan pear image. The module for creating a defect detection model for the Cuiguan pear is used to create a defect detection model for Cuiguan pears based on an improved YOLOv5s, and to set the loss function of the defect detection model. There are multiple versions of YOLOv5 (s,m,l,x), which have similar structures. The main differences are in the two dimensions of depth (number of network layers) and width (number of channels). YOLOv5s is the version with the shallowest depth and the fewest channels, so it is the fastest.

[0094] The Cui Guan pear defect detection model training module is used to divide the dataset into a training set, a validation set, and a test set, and to train, validate, and test the Cui Guan pear defect detection model using the training set, validation set, and test set respectively, and to deploy the Cui Guan pear defect detection model that passes the test. The Cuiguan pear defect detection module is used to acquire real-time images of Cuiguan pears to be detected, input the real-time images of Cuiguan pears into the deployed Cuiguan pear defect detection model, obtain the Cuiguan pear defect detection results, and display the Cuiguan pear defect detection results; The model iteration and optimization module is used to record in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback, and to iteratively optimize the defect detection model of the Cui Guan pear based on the detection log.

[0095] The dataset construction module is specifically used for: A large number of historical images of Cui Guan pears were acquired, and each of the historical Cui Guan pear images was subjected to preprocessing including image enhancement, size normalization, background removal, and format unification. Each historical Cuiguan pear image is smoothed by filtering and then converted to HSV format. A mask image of the region of interest is obtained based on the HSV (hue, saturation, brightness) color range of the Cuiguan pear. Each mask image is then bitwise ANDed with the corresponding historical Cuiguan pear image to segment the initial defect region. The historical Cuiguan pear image carrying the initial defect region is then converted to grayscale and dilated. The dilated historical Cuiguan pear image is segmented using a global thresholding method (binarization is performed, and finally, the resulting binary image is masked again with the original image) to obtain precise defect regions. Labelme data annotation tool is used to label each precise defect region with its defect category. The annotation results are saved in JSON format. The defect categories include at least normal fruit, branch wounds, mechanical damage, and rot. The HSV color space is well-suited to the human visual system, and the ranges of its various color components are clearly defined. Based on the value ranges of each color in the HSV color space, the ranges of each component of the "Cuiguan pear" color are determined. The "Cuiguan pear" color mainly includes yellow and green, with yellow having a smaller value range and green having a larger value range. Therefore, the H, S, and V values ​​of orange are set as the lower limit of the mask color range, while the three green components are set as the upper limit of the mask color range.

[0096] Based on a preset area threshold, small target defects are selected from each of the precise defect regions. After random scaling, random rotation, and random brightness adjustment of each small target, they are randomly copied onto each of the historical green crown pear images (the newly pasted target maintains a distance of at least 10 pixels from the original object to avoid overlap) to perform sample augmentation. A dataset is constructed based on the historical green crown pear images after sample augmentation.

[0097] Although the dataset contains various defect categories of the Cui Guan pear, the class imbalance is addressed from both the data and algorithmic levels. At the data level, images of the minority defect categories are transformed to create more diverse new samples (sample augmentation). This involves applying basic data augmentation operations such as rotation, flipping, scaling, cropping, brightness adjustment, and contrast adjustment to the minority defect category images. Specifically, a GAN (Generative Adversarial Network) is used to generate highly realistic and diverse new images for the minority defect category images, thereby adjusting the distribution of the training data to be closer to balance. At the algorithmic level, the loss function is modified to make the model focus more on the minority class. Specifically, an adjustment factor is used to reduce the loss contribution of "easy-to-classify" samples (usually the majority class), allowing the model to focus more on "hard-to-classify" samples (usually the minority class). The formula is: FL(p_t)=-α_t(1-p_t)^γlog(p_t); Where p_t represents the probability that the model predicts the target class; (1-p_t)^γ represents the modulation factor. When p_t is large (i.e. easy to classify), this modulation factor is close to 0, and the overall loss becomes smaller. That is, by focusing on the parameter γ, the weight of easy-to-classify samples is reduced, so that the model focuses more on difficult-to-classify samples during training. When it is increased (the value of γ is increased), the model will be more "focused" on those "difficult samples" that it has not yet learned and whose classification is ambiguous; α_t represents the class balancing weight.

[0098] In the defect detection model creation module of the Cuiguan pear, the defect detection model of the Cuiguan pear is constructed based on the backbone network, the neck network, and the head. The backbone network is used to extract image features at different levels from the input image of the Cui Guan pear; the neck network is used to fuse multiple image features to obtain fused features; the detection head is used to infer the fused features and output the defect detection results of the Cui Guan pear carrying the defect region and defect category. The defect detection results of the Cuiguan pear also carry confidence and category probability; the role of Neck is to build a powerful feature pyramid that integrates deep high-level semantic features and shallow fine-grained localization features.

[0099] The backbone network is based on a series of sequentially connected layers: a first downsampling layer, a second downsampling layer, a first C3True module (CSP Bottleneck with 3 convolutions, inspired by CSPNet, designed to enhance the learning ability of CNNs by splitting feature maps while reducing computation), a third downsampling layer, a second C3True module, a fourth downsampling layer, a third C3True module, a fifth downsampling layer, a fourth C3True module, an ECA module, and an SPPF module. An ECA module is added before the SPFF module to perform a lightweight channel-level refinement before high-level semantic features enter the Neck. The ECA module has minimal computational overhead, adds almost no latency, but effectively improves feature quality, providing a better starting point for the subsequent Neck.

[0100] C3True enhances the model's representational power by keeping the number of channels constant (rather than reducing dimensionality) in the Bottleneck branch; when the primary goal is to achieve the highest accuracy and the model's speed and size are less of a concern, using a structure that includes C3True is a reasonable choice.

[0101] The SPPF module, or Spatial Pyramid Pooling module, uses multiple max-pooling kernels of different sizes (such as 5x5, 9x9, 13x13) to pool feature maps in parallel, and then concatenates the results. This allows the network to be unrestricted by the input size and greatly increases the receptive field of the backbone features, which helps to detect objects of different scales. SPPF is an accelerated version of SPP, achieving the same effect as SPP by serializing multiple 5x5 pooling layers, but with faster computation.

[0102] The first downsampling layer uses a 6×6 convolution kernel with a stride of 2; the second, third, fourth, and fifth downsampling layers all use a 3×3 convolution kernel with a stride of 2; the first, second, third, and fourth C3True modules are all high-precision C3 modules. The neck network is constructed based on a first feature integration layer, a first UpSample module, a first concat module, a first C3False module, a second feature integration layer, a second UpSample module, a second concat module, a second C3False module, a first CA module, a sixth downsampling layer, a third concat module, a third C3False module, a second CA module, a seventh downsampling layer, a fourth concat module, a fourth C3False module, and a third CA module connected in sequence. The input of the first feature integration layer is connected to the output of the SPPF module; the input of the first concat module is connected to the output of the third C3True module; the input of the second concat module is connected to the output of the second C3True module; the input of the third concat module is connected to the output of the second UpSample module; and the input of the fourth concat module is connected to the output of the first feature integration layer. The neck network uses a two-layer routing mechanism and employs sparse sampling, which preserves more fine-grained details and demonstrates significant superiority in small target detection.

[0103] By combining two different attention mechanisms, ECA (Efficient Channel Attention) and CA (Coordinate Attention), in YOLOv5s, attention is introduced at different macro-level stages of feature extraction. This allows attention mechanisms with different structures and functions to complement each other, enhancing feature representation from multiple dimensions. ECA performs global, lightweight channel preprocessing at the backbone end, improving overall feature quality; CA performs scale-based, fine-grained channel-space joint optimization at the neck output end, particularly improving the localization capability of multi-scale targets. These two attention mechanisms, one preceding and one following, one lightweight and one heavy, one channel and one space, form a complementary and synergistic effect.

[0104] The first and second feature integration layers both use 1×1 convolution kernels with a stride of 1; the sixth and seventh downsampling layers both use 3×3 convolution kernels with a stride of 2; the first, second, third, and fourth C3False modules are all low-precision C3 modules. The core processes of the neck network are: 1. Top-down approach (FPN): The deepest features of the backbone (e.g., 20x20x512) are upsampled and then concatenated with features of the corresponding scale in the backbone (e.g., 40x40x256). This process is repeated to pass high-level semantic information to low-level features.

[0105] 2. Bottom-up (PAN): This is the key to PANet. After completing the top-down fusion, the fused low-level features are downsampled and then concatenated with the fused features from the previous layer. This process passes the precise localization information from the lower layers to the higher-level features.

[0106] Through this bidirectional fusion, PANet enables feature maps at each scale to simultaneously contain rich semantic information and precise location information.

[0107] This improvement is based on YOLOv5s and incorporates the CA attention mechanism. A CA-Net is added to each output of the Neck to jointly refine feature maps of different scales (small, medium, and large targets) using both channel and spatial dimensions. CA attention can capture cross-channel information and directional positional information, which is crucial for target localization, especially helping the model to better outline targets.

[0108] 1. ECA attention mechanism ECA attention mechanism is a channel-based attention mechanism. Its advantage is that it is very lightweight and adds almost no latency, but its disadvantage is that it only focuses on channel relationships.

[0109] ECA is an improved version of the Squeeze-and-Excitation (SE) attention mechanism. The SE attention mechanism "compresses" features through global average pooling, then learns the importance (weights) of each channel through two fully connected layers, and finally "encourages" the original features (channel weighting). The ECA attention mechanism, however, uses one-dimensional convolutions instead of the two fully connected layers in SE, avoiding dimensionality reduction and reconstruction, further reducing the number of parameters and computation while maintaining performance. The core idea of ​​ECA is to capture cross-channel interaction information while avoiding dimensionality reduction. It argues that the dimensionality reduction operation of the fully connected layers in SE attention has side effects on channel attention prediction, therefore using a more efficient, non-dimensionality-reducing method—one-dimensional convolutions—to achieve this.

[0110] It consists of five steps: Step 1: Input feature map: Assume the input feature map size is H×W×C (height×width×number of channels).

[0111] Step 2: Global Average Pooling The spatial information (H×W) of each channel is compressed and converted into a specific numerical value. This step is the same as in the SE module.

[0112] Output: A 1×1×C feature vector, where each value represents global information for one channel.

[0113] Step 3: One-dimensional convolution: This is the innovation and core of the ECA module. Instead of using the two fully connected layers in the SE module, it uses a one-dimensional convolution with a kernel size of k.

[0114] Function: To capture the interaction information between adjacent channels. The kernel size k determines how many of its left and right neighboring channels are considered when calculating attention for each channel. The size of k can be adaptively calculated using a formula.

[0115] Output: A new 1×1×C weight vector after cross-channel information interaction. Note: The dimensions are not changed here; the number of channels C remains unchanged.

[0116] Step 4: Sigmoid activation function: The weight vector output by the one-dimensional convolution is normalized to the range (0,1) to obtain the final attention weight for each channel.

[0117] Step 5: Weighted Allocation for Each Channel The obtained 1×1×C channel attention weights are multiplied channel by channel with the input H×W×C feature map.

[0118] Effect: Important channels are enhanced, while unimportant channels are suppressed.

[0119] Another ingenious design feature of the ECA module is that the kernel size k is not manually set, but adaptively determined based on the number of channels C. There is a mapping relationship between the two: ; in, This indicates taking the nearest odd number; γ and b are hyperparameters, typically set to 2 and 1 respectively. This formula ensures that the coverage of interactions is adaptive for networks of different depths (different number of channels C). Simply put, the more channels there are, the larger the range of interactions (k) that needs to be considered.

[0120] 2. CA Attention Mechanism: CA attention mechanism is a hybrid channel + spatial attention mechanism. Its advantage is that it can simultaneously capture channel relationships and long-range dependencies (with directionality), which is particularly beneficial for target localization. It offers excellent performance with manageable overhead.

[0121] The principle of CA attention mechanism is to decompose channel attention into two parallel 1D global pooling operations along the X and Y directions respectively, thereby capturing the relationship between channels and embedding positional information into the attention graph.

[0122] It consists of three steps: Step 1: Embedding coordinate information: Instead of using global average pooling, CA uses bidirectional pooling to encode global spatial information into a pair of orientation-aware feature maps.

[0123] Input: Feature map X, with dimensions H×W×C.

[0124] X-axis pooling: Uses a pooling kernel of size (H,1) to pool each column of each channel.

[0125] Output: A 1×W×C feature vector Z_h. It captures the global receptive field in the width direction while preserving the positional information in the horizontal direction.

[0126] Y-axis pooling: Uses a pooling kernel of size (1, W) to pool each row of each channel.

[0127] Output: A feature vector Z_w of size H×1×C. It captures the global receptive field in the height direction while preserving the positional information in the vertical direction.

[0128] Step 2: Coordinate Information Fusion and Interaction This is the most ingenious part of CA, which integrates and interacts features from two directions.

[0129] Concatenation and dimensionality reduction: Concatenate Z_h and Z_w, and then perform dimensionality reduction and feature fusion through a shared 1x1 convolution F1 to reduce the number of channels to C / r (r is the reduction rate).

[0130] Nonlinear activation: Use nonlinear activation functions (such as Swish / Sigmoid) to process the fused features.

[0131] Feature splitting: The activated features are split into two independent tensors along the spatial dimension: F_h (size 1×W×C / r) and F_w (size H×1×C / r).

[0132] Convolution and restoration dimensions: Apply 1x1 convolutions F2 and F3 to F_h and F_w respectively to restore the number of channels to the original C, and obtain the intermediate weights G_h and G_w.

[0133] Step 3: Weighted Output The obtained direction-aware weights are applied to the original input feature map.

[0134] Generate attention weights: Apply the Sigmoid function to G_h and G_w respectively to obtain the final attention weights g_h(1×W×C) and g_w(H×1×C).

[0135] Element-wise multiplication: Multiply the weights g_h and g_w element-wise with the original input feature map X.

[0136] g_h weights features in the width direction, emphasizing important column regions.

[0137] g_w weights features in the height direction, emphasizing important row regions.

[0138] Output: The final refined feature map Y is obtained, with dimensions still being H×W×C.

[0139] The detection head is constructed based on a first predictive convolutional layer, a second predictive convolutional layer, and a third predictive convolutional layer; the input of the first predictive convolutional layer is connected to the output of the first CA module; the input of the second predictive convolutional layer is connected to the output of the second CA module; and the input of the third predictive convolutional layer is connected to the output of the third CA module.

[0140] The detection head is the final detection part, receiving feature maps of three different scales from the neck network: large-scale feature map (e.g., 80x80): small receptive field, used to detect small objects; medium-scale feature map (e.g., 40x40): used to detect medium-sized objects; small-scale feature map (e.g., 20x20): large receptive field, used to detect large objects.

[0141] Output for each scale: For each cell of each feature map, the detection head predicts N anchor boxes. For each anchor box, it outputs a vector: [x,y,w,h,obj_score,class1_score,class2_score,...] Where (x,y,w,h) represents the center coordinates and width and height of the bounding box (offsets relative to the cell and anchor point); obj_score represents the confidence score that the bounding box contains an object; class_score represents the probability that the bounding box belongs to each defect category.

[0142] The front part of the detector head has three prediction convolutional layers, which are equivalent to three independent, scale-specific front adapters for the detector head. Their functions are: independent processing: processing the fused feature maps of three different scales in parallel and independently; feature specialization: converting the general fused features into features specifically for bounding box prediction and classification; and channel adjustment: adjusting the number of channels of the feature map to the dimension desired by the detector head.

[0143] They are the key bridges connecting the Feature Pyramid Network (PANet) and the multi-scale prediction head (Detect). In the Neck part, the network outputs feature maps at three different scales (called P3, P4, and P5) through upsampling and downsampling, which are used to detect small, medium, and large objects.

[0144] P3: From a shallower layer, with the largest feature map size (e.g., 80x80), responsible for detecting small objects. P4: From a middle layer, with a medium-sized feature map size (e.g., 40x40), responsible for detecting medium-sized objects. P5: From a deeper layer, with the smallest feature map size (e.g., 20x20), responsible for detecting large objects.

[0145] After feature fusion, the three fused feature maps (P3, P4, P5) cannot be directly fed into the Detect head for prediction. They need to undergo a "refinement" step, which is done by these three independent prediction convolutional layers. Each prediction convolutional layer independently processes a feature map at one scale. Their role is to perform final feature transformation and channel adjustment, specializing the general fused features output by the feature pyramid network into features suitable for the object detection task. The three prediction convolutional layers work in parallel: The first Conv: receives P3 and outputs 80x80x256. This is the final feature map prepared for the small object detection head.

[0146] The second Conv: receives P4 and outputs a 40x40x512 pixel graph. This is the final feature map prepared for the mid-target detection head.

[0147] The third Conv: receives P5 and outputs 20x20x1024. This is the final feature map prepared for a large target detection head.

[0148] Finally, the Detect layer receives these three "refined" feature maps [24,25,26] (i.e., the outputs of the three Conv layers) and performs object detection directly on them (predicted bounding box coordinates, confidence, and class).

[0149] The defect detection model training module for the Cuiguan pear is specifically used for: The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 based on the hold-out method. The defect detection model of the Cui Guan pear is trained using the training set until the loss value of the loss function is less than the preset loss threshold, the validation loss no longer improves, or the maximum epoch is reached. The trained Cui Guan pear defect detection model is validated using the validation set. If the validation fails, the training set is expanded and training continues. If the validation passes, the Cui Guan pear defect detection model is tested using the test set. If the test fails, the training set is expanded and training continues. If the test passes, the Cui Guan pear defect detection model that has passed the test is deployed.

[0150] The model iterative optimization module is specifically used for: The detection log is recorded in real time, including at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback. The detection log is stored and backed up in a distributed manner. Based on a preset iteration cycle, the corresponding detection logs are selected to construct an incremental dataset. The deployed Cuiguan pear defect detection model is trained based on the incremental dataset, and then the deployed Cuiguan pear defect detection model is continuously iterated and optimized.

[0151] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting surface defects in *Pyrus pyrifolia* based on YOLOv5, characterized in that: Includes the following steps: Step S1: Obtain a large number of historical images of Cui Guan pears, and construct a dataset after preprocessing, labeling and sample augmentation of each historical image of Cui Guan pears; Step S2: Create a defect detection model for the Cuiguan pear based on the improved YOLOv5s, and set the loss function of the Cuiguan pear defect detection model; Step S3: Divide the dataset into a training set, a validation set, and a test set. Use the training set, validation set, and test set to train, validate, and test the Cui Guan pear defect detection model, respectively. Deploy the Cui Guan pear defect detection model that passes the test. Step S4: Collect real-time images of the pears to be detected, input the real-time images of the pears into the deployed pear defect detection model, obtain the pear defect detection results, and display the pear defect detection results. Step S5: Record in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback, and iteratively optimize the defect detection model of the Cui Guan pear based on the detection log.

2. The surface defect detection method for *Citrus reticulata* pear based on YOLOv5 as described in claim 1, characterized in that: Step S1 specifically involves: A large number of historical images of Cui Guan pears were acquired, and each of the historical Cui Guan pear images was subjected to preprocessing including image enhancement, size normalization, background removal, and format unification. Each of the historical Cui Guan pear images is filtered and smoothed, then converted to HSV format. A mask image of the region of interest is obtained based on the HSV color range of the Cui Guan pear. Each mask image is then bitwise ANDed with the corresponding historical Cui Guan pear image to segment the initial defect region. The historical Cui Guan pear image carrying the initial defect region is then converted to grayscale and dilated. The dilated historical Cui Guan pear image is then segmented using a global threshold segmentation method to obtain the precise defect region. Each precise defect region is then labeled with a defect category. The defect categories include at least normal fruit, branch wounds, mechanical damage, and rot; Based on a preset area threshold, small target defects are selected from each of the precise defect regions. After random scaling, random rotation, and random brightness adjustment of each small target, they are randomly copied onto each of the historical Cui Guan pear images to perform sample augmentation. A dataset is constructed based on the historical Cui Guan pear images after sample augmentation.

3. The surface defect detection method for *Citrus reticulata* pear based on YOLOv5 as described in claim 1, characterized in that: In step S2, the defect detection model for the Cuiguan pear is constructed based on the backbone network, the neck network, and the detection head. The backbone network is used to extract image features at different levels from the input image of the Cui Guan pear; the neck network is used to fuse the image features to obtain fused features; the detection head is used to infer the fused features and output the defect detection results of the Cui Guan pear carrying the defect region and defect category. The backbone network is based on the sequentially connected first downsampling layer, second downsampling layer, first C3True module, third downsampling layer, second C3True module, fourth downsampling layer, third C3True module, fifth downsampling layer, fourth C3True module, ECA module and SPPF module; The first downsampling layer uses a 6×6 convolution kernel with a stride of 2; the second, third, fourth, and fifth downsampling layers all use a 3×3 convolution kernel with a stride of 2; the first, second, third, and fourth C3True modules are all high-precision C3 modules. The neck network is constructed based on a first feature integration layer, a first UpSample module, a first concat module, a first C3False module, a second feature integration layer, a second UpSample module, a second concat module, a second C3False module, a first CA module, a sixth downsampling layer, a third concat module, a third C3False module, a second CA module, a seventh downsampling layer, a fourth concat module, a fourth C3False module, and a third CA module connected in sequence. The input of the first feature integration layer is connected to the output of the SPPF module; the input of the first concat module is connected to the output of the third C3True module; the input of the second concat module is connected to the output of the second C3True module; the input of the third concat module is connected to the output of the second UpSample module; and the input of the fourth concat module is connected to the output of the first feature integration layer. The first and second feature integration layers both use 1×1 convolution kernels with a stride of 1; the sixth and seventh downsampling layers both use 3×3 convolution kernels with a stride of 2; the first, second, third, and fourth C3False modules are all low-precision C3 modules. The detection head is constructed based on a first predictive convolutional layer, a second predictive convolutional layer, and a third predictive convolutional layer; the input of the first predictive convolutional layer is connected to the output of the first CA module; the input of the second predictive convolutional layer is connected to the output of the second CA module; and the input of the third predictive convolutional layer is connected to the output of the third CA module.

4. The surface defect detection method for *Citrus reticulata* pear based on YOLOv5 as described in claim 1, characterized in that: Step S3 specifically involves: The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 based on the hold-out method. The defect detection model of the Cui Guan pear is trained using the training set until the loss value of the loss function is less than the preset loss threshold, the validation loss no longer improves, or the maximum epoch is reached. The trained Cui Guan pear defect detection model is validated using the validation set. If the validation fails, the training set is expanded and training continues. If the validation passes, the Cui Guan pear defect detection model is tested using the test set. If the test fails, the training set is expanded and training continues. If the test passes, the Cui Guan pear defect detection model that has passed the test is deployed.

5. The surface defect detection method for *Citrus reticulata* pear based on YOLOv5 as described in claim 1, characterized in that: Step S5 specifically involves: The system records in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the feedback of the detection error. The detection log is then stored and backed up in a distributed manner. Based on a preset iteration cycle, the corresponding detection logs are selected to construct an incremental dataset. The deployed Cuiguan pear defect detection model is trained based on the incremental dataset, and then the deployed Cuiguan pear defect detection model is continuously iterated and optimized.

6. A surface defect detection system for the 'Crimson Crown' pear based on YOLOv5, characterized in that: Includes the following modules: The dataset construction module is used to acquire a large number of historical images of Cui Guan pears, and to construct a dataset after preprocessing, labeling and expanding the samples of each historical Cui Guan pear image. The module for creating a defect detection model for Cuiguan pear is used to create a defect detection model for Cuiguan pear based on the improved YOLOv5s and to set the loss function of the defect detection model for Cuiguan pear. The Cui Guan pear defect detection model training module is used to divide the dataset into a training set, a validation set, and a test set, and to train, validate, and test the Cui Guan pear defect detection model using the training set, validation set, and test set respectively, and to deploy the Cui Guan pear defect detection model that passes the test. The Cuiguan pear defect detection module is used to acquire real-time images of Cuiguan pears to be detected, input the real-time images of Cuiguan pears into the deployed Cuiguan pear defect detection model, obtain the Cuiguan pear defect detection results, and display the Cuiguan pear defect detection results; The model iteration and optimization module is used to record in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the detection error feedback, and to iteratively optimize the defect detection model of the Cui Guan pear based on the detection log.

7. The YOLOv5-based surface defect detection system for *Cinnamomum camphora* pears as described in claim 6, characterized in that: The dataset construction module is specifically used for: A large number of historical images of Cui Guan pears were acquired, and each of the historical Cui Guan pear images was subjected to preprocessing including image enhancement, size normalization, background removal, and format unification. Each of the historical Cui Guan pear images is filtered and smoothed, then converted to HSV format. A mask image of the region of interest is obtained based on the HSV color range of the Cui Guan pear. Each mask image is then bitwise ANDed with the corresponding historical Cui Guan pear image to segment the initial defect region. The historical Cui Guan pear image carrying the initial defect region is then converted to grayscale and dilated. The dilated historical Cui Guan pear image is then segmented using a global threshold segmentation method to obtain the precise defect region. Each precise defect region is then labeled with a defect category. The defect categories include at least normal fruit, branch wounds, mechanical damage, and rot; Based on a preset area threshold, small target defects are selected from each of the precise defect regions. After random scaling, random rotation, and random brightness adjustment of each small target, they are randomly copied onto each of the historical Cui Guan pear images to perform sample augmentation. A dataset is constructed based on the historical Cui Guan pear images after sample augmentation.

8. The YOLOv5-based surface defect detection system for *Cinnamomum camphora* pears as described in claim 6, characterized in that: In the defect detection model creation module for the Cuiguan pear, the defect detection model for the Cuiguan pear is constructed based on the backbone network, the neck network, and the detection head; The backbone network is used to extract image features at different levels from the input image of the Cui Guan pear; the neck network is used to fuse the image features to obtain fused features; the detection head is used to infer the fused features and output the defect detection results of the Cui Guan pear carrying the defect region and defect category. The backbone network is based on the sequentially connected first downsampling layer, second downsampling layer, first C3True module, third downsampling layer, second C3True module, fourth downsampling layer, third C3True module, fifth downsampling layer, fourth C3True module, ECA module and SPPF module; The first downsampling layer uses a 6×6 convolution kernel with a stride of 2; the second, third, fourth, and fifth downsampling layers all use a 3×3 convolution kernel with a stride of 2; the first, second, third, and fourth C3True modules are all high-precision C3 modules. The neck network is constructed based on a first feature integration layer, a first UpSample module, a first concat module, a first C3False module, a second feature integration layer, a second UpSample module, a second concat module, a second C3False module, a first CA module, a sixth downsampling layer, a third concat module, a third C3False module, a second CA module, a seventh downsampling layer, a fourth concat module, a fourth C3False module, and a third CA module connected in sequence. The input of the first feature integration layer is connected to the output of the SPPF module; the input of the first concat module is connected to the output of the third C3True module; the input of the second concat module is connected to the output of the second C3True module; the input of the third concat module is connected to the output of the second UpSample module; and the input of the fourth concat module is connected to the output of the first feature integration layer. The first and second feature integration layers both use 1×1 convolution kernels with a stride of 1; the sixth and seventh downsampling layers both use 3×3 convolution kernels with a stride of 2; the first, second, third, and fourth C3False modules are all low-precision C3 modules. The detection head is constructed based on a first predictive convolutional layer, a second predictive convolutional layer, and a third predictive convolutional layer; the input of the first predictive convolutional layer is connected to the output of the first CA module; the input of the second predictive convolutional layer is connected to the output of the second CA module; and the input of the third predictive convolutional layer is connected to the output of the third CA module.

9. The YOLOv5-based surface defect detection system for *Cinnamomum camphora* pears as described in claim 6, characterized in that: The defect detection model training module for the Cuiguan pear is specifically used for: The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1 based on the hold-out method. The defect detection model of the Cui Guan pear is trained using the training set until the loss value of the loss function is less than the preset loss threshold, the validation loss no longer improves, or the maximum epoch is reached. The trained Cui Guan pear defect detection model is validated using the validation set. If the validation fails, the training set is expanded and training continues. If the validation passes, the Cui Guan pear defect detection model is tested using the test set. If the test fails, the training set is expanded and training continues. If the test passes, the Cui Guan pear defect detection model that has passed the test is deployed.

10. The YOLOv5-based surface defect detection system for *Cinnamomum camphora* pears as described in claim 6, characterized in that: The model iterative optimization module is specifically used for: The system records in real time a detection log that includes at least the detection time, real-time images of the Cui Guan pear, the defect detection results of the Cui Guan pear, and the feedback of the detection error. The detection log is then stored and backed up in a distributed manner. Based on a preset iteration cycle, the corresponding detection logs are selected to construct an incremental dataset. The deployed Cuiguan pear defect detection model is trained based on the incremental dataset, and then the deployed Cuiguan pear defect detection model is continuously iterated and optimized.

Images