Apple surface defect detection method and system based on deep learning
Through the improved convolutional neural network and deep residual network combined with attention mechanism, combined with transfer learning and generative adversarial network, the problems of low efficiency and poor robustness in Apple's surface defect detection are solved, and high-precision and low-cost automated detection are achieved.
Patent Information
- Application Number
- CN202510437023.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-07-25
AI Technical Summary
The prior art has low efficiency and is susceptible to human factors in Apple's surface defect detection. Traditional image processing algorithms are difficult to identify small defects under complex backgrounds and different lighting conditions, and are overly dependent on labeled data.
The improved convolutional neural network is used to combine deep residual networks and attention mechanisms, combine transfer learning and generative adversarial networks, and perform multi-scale feature extraction and data enhancement, identify defect locations and types through bounding box regression and semantic segmentation, and classify them using a decision tree or a support vector machine.
It improves the accuracy and adaptability of Apple's surface defect detection, reduces detection time and cost, enhances the stability and robustness of the system, can adapt to complex lighting and background conditions, and has the ability to continuously optimize.
Smart Images
Figure CN120375360A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical fields of machine vision and deep learning, and more particularly to an apple surface defect detection method and system based on deep learning. Background Art
[0002] As a common fruit, the detection of apple surface defects has always been an important link in fruit quality control. Traditional apple surface defect detection methods usually rely on manual visual inspection or automated detection systems based on traditional image processing algorithms. These methods have several obvious defects: First, manual inspection is not only inefficient but also easily affected by human factors; Second, traditional image processing algorithms are difficult to accurately identify tiny defects on apples under complex backgrounds and different lighting conditions. Therefore, there is an urgent need in the prior art for a more efficient, intelligent, and highly robust automated detection method.
[0003] Image processing methods based on deep learning have demonstrated powerful capabilities in image classification and object detection, but still face some technical challenges in the detection of fruit surface defects. The main problems include:
[0004] There are many types of apple surface defects and they have a certain degree of variability, which brings difficulties to training the model.
[0005] The surface features of apples are easily affected by the external environment (such as lighting, background, viewing angle, etc.), and the robustness of traditional deep learning models is poor.
[0006] Traditional deep learning methods require a large amount of labeled data for training, and the labeling of apple surface defects is often time-consuming and difficult.
[0007] Therefore, how to design an innovative deep learning network architecture, optimize the training process, and a specific detection process for apple surface defects has become an urgent problem to be solved by those skilled in the art. Summary of the Invention
[0008] In view of this, the present invention provides an apple surface defect detection method and system based on deep learning, which can solve the problem that when detecting apple surface defects in the prior art, the adaptability to the unique features of the apple surface is not strong, resulting in unsatisfactory detection effects and low detection accuracy.
[0009] To achieve the above object, the present invention adopts the following technical solutions:
[0010] In the first aspect, an embodiment of the present invention provides an apple surface defect detection method based on deep learning, including the following steps:
[0011] S1. Collect the image data of apple surface defects and perform preprocessing; the image data includes images of five types, namely healthy, mechanical damage, pests and diseases, wrinkles, and rot, under various lighting conditions and different backgrounds.
[0012] S2. Use an improved convolutional neural network combined with a deep residual network and an attention mechanism to perform multi-scale feature extraction on the apple surface images and extract the apple surface defect features.
[0013] S3. Based on the extracted apple surface defect features, use transfer learning to fine-tune the pre-trained model and train the apple surface defect detection model.
[0014] S4. Use the trained apple surface defect detection model to analyze the image data of the apples to be detected and identify the defect positions and types on the apple surface.
[0015] S5. Generate corresponding defect reports according to the recognition results and classify or reject the defective apples.
[0016] Further, the preprocessing in step S1 includes:
[0017] a) Image enhancement: Use histogram equalization, adaptive contrast adjustment, or local brightness correction methods for image enhancement.
[0018] b) Noise filtering: Use Gaussian filtering, median filtering, or bilateral filtering to remove image noise.
[0019] c) Size normalization: Adjust all input images to a unified size for subsequent processing.
[0020] Further, in step S2, the improved convolutional neural network includes:
[0021] Multi-scale feature extraction module: Adopt convolutional layers and pyramid pooling structures of different scales to ensure the extraction of different scale information from details to the whole of the apple surface.
[0022] Fusion multi-scale feature module: Adopt an adaptive fusion strategy to adaptively select the fusion level according to different defect types and enhance the sensitivity of the network to specific types of defects.
[0023] The deep residual network in step S2: Introduce a residual connection through ResNet to directly add the input of a certain layer in the multi-scale feature extraction module to the output.
[0024] The attention mechanism in step S2: The spatial attention mechanism helps the network weight different regions according to the feature importance at different positions in the image, focusing on the positions where surface defects may occur; and the channel attention mechanism allows the network to automatically select important feature channels in the image, thereby enhancing the model's defect recognition ability.
[0025] Further, step S3 includes:
[0026] Based on the extracted apple surface defect features, during the transfer learning process, select the ResNet50 model pre-trained on the large-scale image dataset ImageNet and fine-tune based on this model;
[0027] During the fine-tuning process, adopt a phased fine-tuning strategy, first freeze the lower-level convolutional layers, and gradually unfreeze the higher-level network for training to avoid overfitting.
[0028] Further, in step S4, identifying the defect positions and types on the apple surface includes:
[0029] During the defect recognition process, use bounding box regression or semantic segmentation to accurately locate the defect positions and mark the specific coordinates of the defects;
[0030] And adopt multi-task learning to perform defect classification and localization simultaneously to improve the detection efficiency.
[0031] Further, step S5 includes:
[0032] Generate information including defect type, defect size, and defect area according to the recognition result, and generate a quality score for each apple;
[0033] And adopt a classification strategy based on decision tree or support vector machine to classify apples according to the defect type and severity.
[0034] Further, it also includes:
[0035] S6. Receive the recognition result that corrects the errors in the defect report as the corrected training data, and combine it with new defect samples to retrain the apple surface defect detection model regularly.
[0036] In a second aspect, an embodiment of the present invention also provides a deep learning-based apple surface defect detection system, including:
[0037] An acquisition and preprocessing module for acquiring image data of apple surface defects and performing preprocessing; the image data includes: images of five types, namely healthy, mechanical damage, pests and diseases, wrinkles, and rot, under various lighting conditions and different backgrounds;
[0038] A feature extraction module, which is used to perform multi-scale feature extraction on apple surface images by using an improved convolutional neural network combined with a deep residual network and an attention mechanism, and extract apple surface defect features;
[0039] A training module, which is used to fine-tune a pre-trained model based on the extracted apple surface defect features by using transfer learning, and train an apple surface defect detection model;
[0040] An identification module, which is used to analyze the image data of the apple to be detected by using the trained apple surface defect detection model, and identify the defect positions and types on the apple surface;
[0041] A report generation module, which is used to generate corresponding defect reports according to the identification results, and classify or reject defective apples.
[0042] In a third aspect, an embodiment of the present invention further provides a device, including:
[0043] At least one processor; and a memory communicatively connected to the at least one processor;
[0044] Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute any one of the apple surface defect detection methods based on deep learning in the first aspect embodiments.
[0045] In a fourth aspect, an embodiment of the present invention further provides a storage medium, in which instructions are stored, and when the instructions are run on a terminal, any one of the apple surface defect detection methods based on deep learning in the first aspect embodiments can be implemented.
[0046] The descriptions of the second to fourth aspects in the present invention can refer to the detailed descriptions of the first aspect; and the beneficial effects of the descriptions of the second to fourth aspects can refer to the beneficial effect analysis of the first aspect, which will not be elaborated here.
[0047] It can be seen from the above technical solutions that, compared with the prior art, the present invention has the following advantages:
[0048] The accuracy and adaptability of apple surface defect detection are improved; the detection speed is significantly increased, and the detection time and cost are reduced; in addition, the stability and robustness of the system are enhanced, and the detection accuracy is improved; it can adapt to complex lighting and background conditions, and has the ability of continuous optimization, and has high practical value and market application prospects. Description of the Drawings
[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.
[0050] Figure 1 Flowchart of the apple surface defect detection method based on deep learning provided by the present invention.
[0051] Figure 2 Schematic diagram of identifying the defect positions on the apple surface provided by the present invention.
[0052] Figure 3 Block diagram of the apple surface defect detection system based on deep learning provided by the present invention. Detailed implementation manners
[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.
[0054] Embodiment 1:
[0055] Refer to Figure 1 As shown, the embodiments of the present invention disclose a method for detecting apple surface defects based on deep learning, including the following steps S1 to S6:
[0056] S1. Collect image data of apple surface defects and perform preprocessing; the image data includes: images of five types, namely healthy, mechanical damage, pests and diseases, wrinkles, and rot, under various lighting conditions and different backgrounds.
[0057] First, collect image data of the apple surface under different lighting conditions (such as normal lighting, low lighting, high lighting, etc.) and different backgrounds (such as black, white, natural background, etc.). The collected image data contains five types of defects: healthy, mechanical damage, pests and diseases, wrinkles, and rot. When collecting images, a high-definition imaging device is used to ensure that the images have sufficient resolution.
[0058] After image collection, preprocessing is performed, which mainly includes the following aspects:
[0059] Image Enhancement: Methods such as histogram equalization, adaptive contrast adjustment, or local brightness correction are used to improve the visibility of the image and enhance the defect information in the image.
[0060] Noise Filtering: Techniques such as Gaussian filtering, median filtering, or bilateral filtering are used to remove the noise in the image and retain important image features.
[0061] Size Normalization: All images are uniformly adjusted to the same size (e.g., 224×224 pixels) for subsequent processing.
[0062] Among them, the above image enhancement can also be based on a generative adversarial network (GAN) to achieve data augmentation for apple surface defect images; in apple surface defect detection, for example, due to the scarcity of actual defect samples, data augmentation can be effectively improved by using GAN to generate images with realistic apple defects for training the deep learning model.
[0063] The generative adversarial network includes:
[0064] a Generator: Responsible for generating fake apple defect images. It takes random noise (usually a low-dimensional vector) as input and converts it into a realistic image. The goal of the generator is to make the generated image indistinguishable from a real one by the discriminator, that is, to be able to "fool" the discriminator.
[0065] b Discriminator: Responsible for distinguishing whether the image is a real apple surface image. It takes real images and images generated by the generator as input and outputs a value representing the probability that the image is a real image. The goal of the discriminator is to accurately distinguish the difference between real images and generated images.
[0066] The images generated by the generator can be used as "virtual" samples and added to the dataset of real collected images. In this way, the diversity of the dataset can be effectively increased, especially in the case of scarce defect types, which can significantly improve the training effect of the model.
[0067] By training a GAN model, it is possible to generate apple images of various defect types, especially some defects that are difficult to capture (such as minor pests and diseases, early rot, etc.). Generate images under different lighting and background conditions: In actual shooting, the lighting conditions and background often change, while the GAN model can simulate different shooting environments and generate apple images under various lighting and background conditions. Generate deformed images: By adjusting the noise input vector, images with different morphological and severity defects on the surface can be generated to expand the dataset and improve the model's recognition ability for diverse defects. Generate diverse image samples: The generator can generate diverse image samples based on diverse input noises, including different defect types, different defect shapes, different apple varieties, different resolutions, etc., effectively avoiding the problem of sample imbalance in the training dataset.
[0068] Finally, by generating images with realistic apple defects through a generative adversarial network (GAN), the training dataset can be effectively expanded, enriched, and the information loss caused by traditional data augmentation methods (such as rotation, cropping, flipping, etc.) can be effectively avoided, improving the training effect and generalization ability of the apple surface defect detection model.
[0069] S2. Use an improved convolutional neural network combined with a deep residual network and an attention mechanism to perform multi-scale feature extraction on the apple surface image and extract apple surface defect features;
[0070] In this step, specifically, an improved convolutional neural network (CNN) is adopted, which combines a deep residual network (ResNet50) and an attention mechanism to enhance the diversity and robustness of feature extraction. Through multi-scale feature extraction, defects of different sizes can be captured, and misdetection caused by the curvature of the apple surface or uneven lighting can be effectively avoided.
[0071] In the network architecture, the introduction of the attention mechanism enables the model to automatically focus on the most critical areas of the apple surface (such as defects like cracks and black dots on the peel), thus improving the model's robustness in complex backgrounds.
[0072] Among them, the improved convolutional neural network (CNN) includes: a multi-scale feature extraction module and a multi-scale feature fusion module:
[0073] Multi-scale Feature Extraction Module: Different-sized convolutional kernels (1x1, 3x3, 5x5) of convolutional layers with different scales and pyramid pooling structures (such as PSPNet) are adopted to ensure the extraction of different-scale information from the details to the whole of the apple surface. For example, a high-resolution convolutional layer is used to extract small defects on the apple surface, and a low-resolution convolutional layer is used to capture larger defects. Additionally, depthwise separable convolution and dilated convolution can be combined to design a lightweight feature extraction layer, and color space conversion (such as Lab color space) is introduced to enhance the extraction of color features.
[0074] Multi-scale Feature Fusion Module: In the feature fusion stage, an adaptive fusion strategy can be adopted, that is, the fusion level is adaptively selected according to different defect types to enhance the sensitivity of the network to specific types of defects (such as fine cracks, large spots, etc.).
[0075] Deep Residual Network Module: Residual connections are introduced through ResNet50, and the input of a certain layer in the multi-scale feature extraction module is directly added to the output to enhance the learning ability of the model. It solves the problems of gradient disappearance and information loss during the training of deep networks. Through these connections, the network can be trained more easily and can capture more complex features. ResNet50 makes the information "jump" between multiple layers instead of being transmitted layer by layer by directly adding the input of a certain layer to the output. This structure can effectively alleviate the problem of gradient disappearance in the training of deep networks, enabling the network to better capture complex high-level features. In the detection of apple surface defects, the use of a deep residual network can help the model capture more details, especially having better detection ability for fine defects (such as tiny scratches or spots).
[0076] Attention Mechanism Module: In image classification or object detection, the attention mechanism allows the network to automatically focus on the most important regions when processing images, such as defects on the apple surface (e.g., cracks, rotten spots, spots, etc.), while ignoring other unimportant background regions.
[0077] Spatial Attention Mechanism: This mechanism helps the network weight different regions according to the importance of features at different positions in the image, so as to focus on the positions where surface defects may appear.
[0078] Channel Attention Mechanism: This mechanism allows the network to automatically select important feature channels in the image (for example, the red, green, and blue channels may have different degrees of attention), thereby improving the model's ability to recognize defects.
[0079] In the apple surface defect detection task, the network combined with the attention mechanism can automatically focus on the key parts of the apple surface (such as crack or spot areas), thereby reducing interference from irrelevant backgrounds and improving the detection accuracy.
[0080] Therefore, in this step, when performing apple surface defect detection, the improved convolutional neural network (CNN) will first perform multi-scale feature extraction on the image to extract features at different levels and scales. Next, the deep residual network (ResNet50) will further extract complex and high-level features through its skip connection mechanism, which is crucial for judging small defects (such as tiny cracks or spots). Finally, the attention mechanism in the network will guide the model to focus on the defect areas on the apple surface, thereby improving the detection accuracy.
[0081] S3. Extract features of apple surface defects; use transfer learning to fine-tune the pre-trained model and train the apple surface defect detection model.
[0082] In this step, select the pre-trained model: The source and type of the selected pre-trained model can be specified in detail. For example, select the ResNet50 model pre-trained on a large-scale image dataset (such as ImageNet). This model has already learned general image features and is suitable for the apple surface defect detection task.
[0083] By adopting the transfer learning technology, use the existing general image recognition network for pre-training, and then fine-tune it for the specific task of apple surface defects, thus greatly reducing the need for a large amount of labeled data. Adopt a phased fine-tuning strategy. First, freeze the lower-level convolutional layers of the pre-trained model and only train the higher-level network to help the model quickly adapt to the target task. Then, gradually unfreeze some of the lower-level networks to refine the model and avoid overfitting.
[0084] In the task of apple surface defect detection, the application of transfer learning can be operated according to the following steps:
[0085] 1) Select the pre-trained model: Select a model that has been trained on a large-scale dataset such as ImageNet (for example, ResNet50).
[0086] 2) Freeze the lower-level convolutional layers: The lower-level convolutional layers of the ImageNet model have learned some general features (such as edges, textures, etc.), which are also useful in apple defect detection. Therefore, these lower-level convolutional layers can be frozen to prevent their weights from being updated during the fine-tuning process.
[0087] 3) Replace the top fully connected layer: Since the recognition of apple surface defects requires specific feature extraction (such as scratches, black spots, etc.), the original fully connected layer needs to be replaced with a network structure suitable for the target task (such as a fully connected layer adapted to apple surface defect classification).
[0088] 4) Fine-tune the fully connected layer: Train the new fully connected layer with a small amount of apple surface image data. Since the convolutional layer pre-trained on a large-scale dataset has been used, only a small amount of data is needed to achieve good results when fine-tuning for the new task.
[0089] 5) Final training: Train the fine-tuned model with a small amount of apple image data to adjust the network parameters to better adapt to the characteristics of apple surface defects.
[0090] Therefore, in this step, since the large-scale dataset on the source task has trained the model, transfer learning can utilize this pre-trained knowledge in the target task, reducing the need for a large amount of labeled data for the target task. And by leveraging the existing knowledge, the training time can be reduced because the model has learned the basic features and can adapt to the target task faster. In the case of a small amount of target task data, transfer learning can significantly improve the model's performance because it avoids the overfitting problem caused by training from scratch. It is an effective means to solve problems such as insufficient labeled data, long training time, and poor model performance in apple surface defect detection.
[0091] S4. Use the trained apple surface defect detection model to analyze the image data of the apples to be detected, and identify the defect positions and types on the apple surface;
[0092] The improved convolutional neural network (CNN) will first perform multi-scale feature extraction on the image to extract features at different levels and scales. Next, the deep residual network (ResNet) will further extract complex and high-level features through its skip connection mechanism, which is crucial for detecting small defects (such as tiny cracks or spots). Finally, the attention mechanism in the network will guide the model to focus on the defect areas on the apple surface, thereby improving the detection accuracy.
[0093] In this step, by combining bounding box regression or semantic segmentation techniques, not only the defect types can be identified, but also the positions of the defects can be accurately located, giving the precise coordinates of the defects, as shown in Figure 2 shown. Multi-task learning can be adopted to simultaneously perform the defect classification and localization tasks in the same model. By sharing the underlying features, the detection efficiency can be improved.
[0094] S5. Generate a corresponding defect report based on the recognition results, and classify or reject the defective apples. In this step, the generated defect report can include detailed information such as defect types, defect sizes, and defect areas, and provide a quality score for the apples. This information not only helps with manual intervention but also supports quality management on the automated production line.
[0095] In addition, the apple classification adopts a classification strategy based on decision trees or support vector machines (SVMs), combines the defect types and severity levels of apples, and automatically classifies apples into three categories: "high-quality", "qualified", or "rejected", supporting intelligent classification.
[0096] S6. Receive the recognition results of correcting errors in the defect report as the corrected training data, and combine new defect samples to retrain the apple surface defect detection model regularly.
[0097] In this step, according to the corrected defect recognition results collected in real time (such as misjudged samples or incorrect recognition results), as new training data, the apple surface defect detection model is retrained regularly to improve the accuracy and robustness of the model.
[0098] Through the combination of a deep residual network and an attention mechanism, the present invention can efficiently detect defects on the apple surface under variable lighting and background conditions, and the detection accuracy is significantly improved compared with traditional methods. It has strong robustness. Through multi-scale feature extraction and data augmentation techniques, the present invention can effectively handle defects of different sizes and forms, avoiding the performance degradation of traditional methods when dealing with complex textures and deformations on the apple surface. The data augmentation method based on transfer learning and generative adversarial networks significantly reduces the dependence on a large amount of labeled data, making this method have a low training cost and high adaptability in practical applications. Through the specific detection optimization process of apple surface defects, the present invention can be applied to automated detection, reducing manual intervention and helping to improve the working efficiency of the production line.
[0099] Example 2:
[0100] The embodiment of the present invention also provides a deep learning-based apple surface defect detection system. As shown in Figure 3 , it includes:
[0101] An acquisition and preprocessing module, which is used to acquire image data of apple surface defects and perform preprocessing; the image data includes images of five types: healthy, mechanical damage, pests and diseases, wrinkles, and rot under various lighting conditions and different backgrounds.
[0102] This module is responsible for collecting apple surface image data and preprocessing the collected images to ensure that the input data is suitable for subsequent deep learning analysis. The collected image data includes five types of images of apples under different lighting conditions (e.g., low light, high light, normal light) and different backgrounds (e.g., natural background, black background, white background, etc.), including defect types such as healthy, mechanical damage, pests and diseases, wrinkles, and rot. The preprocessing steps include image enhancement (increasing image contrast and brightness), noise removal (such as Gaussian filtering, median filtering, etc.), and size normalization (adjusting all images to the same size for subsequent processing).
[0103] Feature extraction module, which uses an improved convolutional neural network combined with a deep residual network and an attention mechanism to perform multi-scale feature extraction on apple surface images and extract apple surface defect features. This module uses an improved convolutional neural network (CNN) combined with a deep residual network (ResNet50) and an attention mechanism to extract multi-scale features from apple surface images and identify the defect features on the apple surface.
[0104] Among them, the convolutional neural network (CNN): is used to extract low-level features of images (such as edges, textures, etc.). The deep residual network (ResNet): improves the depth and accuracy of feature extraction through residual connections, avoiding the problem of gradient disappearance. The attention mechanism: guides the network to focus on the positions and features where defects may exist on the apple surface through spatial and channel attention mechanisms, further improving the detection accuracy.
[0105] Training module, which is used to extract apple surface defect features; use transfer learning to fine-tune the pre-trained model and train an apple surface defect detection model; this module is responsible for fine-tuning the pre-trained model using transfer learning based on the extracted apple surface defect features and training a deep learning model specific to apple surface defect detection. Through transfer learning technology, a deep neural network model pre-trained on a large-scale dataset (such as ImageNet) is used as a basis and further fine-tuned to adapt to the specific task of apple surface defects. A phased fine-tuning strategy is adopted, freezing the lower-level convolutional layers first and gradually unfreezing the higher-level network to avoid overfitting and improve the generalization ability of the model.
[0106] Recognition module, which uses the trained apple surface defect detection model to analyze the image data of the apple to be detected and identify the defect positions and types on the apple surface; this module uses the trained apple surface defect detection model to analyze the image of the apple to be detected and identify the defect positions and types on the apple surface in the image. The recognition module uses bounding box regression (accurately locating the defect positions) and semantic segmentation (pixel-level annotation of the defect areas), and simultaneously performs defect classification and localization to improve the detection efficiency and accuracy.
[0107] A report generation module is used to generate corresponding defect reports based on the recognition results, and classify or reject defective apples. This module generates defect reports according to the recognition results and classifies or rejects apples. The reports include detailed information such as defect types (such as mechanical damage, pests and diseases, rot, etc.), the size and location of the defects. According to the severity of the defects, the apples are classified, or the defective apples are rejected, so as to ensure that only apples meeting the quality standards enter the market.
[0108] Through the cooperation of multiple functional modules, this system can efficiently detect and classify apple surface defects under different conditions, greatly improving the detection accuracy and efficiency. In the quality control of apples after picking, using the detection system of the present invention to automatically identify and classify apple defects, the results show that the system has greatly improved the classification efficiency and reduced human errors. The present invention can effectively improve the detection accuracy and efficiency of apple surface defects through an improved convolutional neural network, deep residual network and attention mechanism, providing an intelligent and efficient solution for agricultural production and quality control.
[0109] Example 3:
[0110] The embodiment of the present invention further provides a device, including:
[0111] At least one processor: The device includes at least one processor (usually a CPU or GPU), which is responsible for performing various computing tasks, such as image processing, feature extraction, model inference, etc. The processor is the core component for performing the apple surface defect detection task.
[0112] Memory: The device includes a memory (such as RAM, hard disk, SSD, etc.) communicatively connected to the processor, which is used to store information such as instructions, data, and trained models. The instructions in the memory are executed by the processor, thereby implementing each step of the apple surface defect detection method.
[0113] Instructions executable by the processor are stored in the memory. The instruction set includes various operation steps related to the apple surface defect detection method, such as image acquisition, preprocessing, feature extraction, model training, defect recognition, and report generation. It can implement each technical function of the apple surface defect detection method based on deep learning in Example 1, thereby completing the efficient detection, recognition, and classification of apple surface defects.
[0114] This device provides a hardware platform that can execute the apple surface defect detection method based on deep learning, with advantages such as fast processing speed and high recognition accuracy, and is suitable for large-scale apple quality detection tasks.
[0115] Example 4:
[0116] Another embodiment of the present invention further provides a storage medium, which can be any data storage device capable of storing computer instructions, such as a USB flash drive, a hard disk, a solid-state drive (SSD), an optical disc, cloud storage, etc.
[0117] The storage medium contains a set of executable computer program instructions.
[0118] Instruction content:
[0119] The instruction set stored in the storage medium includes all the key steps of the apple surface defect detection method, such as image acquisition and preprocessing, feature extraction, model training and fine-tuning, defect recognition, report generation, etc.
[0120] These instructions can guide the terminal device (such as a processor) to execute tasks in sequence and gradually complete the detection of apple surface defects.
[0121] When the instructions in the storage medium are loaded and executed by a terminal device (such as a computer, an embedded device, a server, etc.), the device can implement various functions in the apple surface defect detection method of Embodiment 1, including image analysis, defect recognition, defect report generation, etc.
[0122] Through the storage medium, users can save and deploy the deep learning model and the detection method to multiple terminal devices, facilitating the detection of apple surface defects on different devices. This technical solution can greatly improve the portability and flexibility of the method, enabling the apple surface defect detection system to run on different hardware platforms and facilitating software update and maintenance.
[0123] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.
[0124] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An apple surface defect detection method based on deep learning, characterized in that, It includes the following steps: S1. Collect image data of apple surface defects and perform preprocessing; the image data includes images of five types, namely healthy, mechanical damage, pests and diseases, wrinkles, and rot, under various lighting conditions and different backgrounds; S2. Use an improved convolutional neural network combined with a deep residual network and an attention mechanism to perform multi-scale feature extraction on the apple surface image and extract apple surface defect features; S3. Based on the extracted apple surface defect features, use transfer learning to fine-tune a pre-trained model and train an apple surface defect detection model; S4. Use the trained apple surface defect detection model to analyze the image data of the apple to be detected and identify the defect positions and types on the apple surface; S5. Generate corresponding defect reports according to the recognition results and classify or reject defective apples.
2. The apple surface defect detection method based on deep learning according to claim 1, characterized in that, The preprocessing in step S1 includes: a) Image enhancement: Use histogram equalization, adaptive contrast adjustment, or local brightness correction methods for image enhancement; b) Noise filtering: Use Gaussian filtering, median filtering, or bilateral filtering to remove image noise; c) Size normalization: Adjust all input images to a unified size for subsequent processing.
3. A method for detecting apple surface defects based on deep learning according to claim 1, characterized in that In step S2, the improved convolutional neural network includes: Multi-scale feature extraction module: Adopt convolutional layers and pyramid pooling structures of different scales to ensure that different scale information from details to the whole of the apple surface is extracted; Fusion multi-scale feature module: Adopt an adaptive fusion strategy to adaptively select the fusion level according to different defect types and enhance the network's sensitivity to specific types of defects; The deep residual network in step S2: Introduce a residual connection through ResNet to directly add the input of a certain layer in the multi-scale feature extraction module to the output; The attention mechanism in step S2: Help the network weight different regions according to the importance of features in different positions in the image through a spatial attention mechanism, focusing on the positions where surface defects may appear; and allow the network to automatically select important feature channels in the image through a channel attention mechanism, thereby improving the model's defect recognition ability.
4. A method for detecting apple surface defects based on deep learning according to claim 1, characterized in that, Step S3 includes: Based on the extracted apple surface defect features, in the process of transfer learning, select the ResNet50 model pre-trained on the large-scale image dataset ImageNet and fine-tune based on this model; During the fine-tuning process, adopt a phased fine-tuning strategy, first freeze the lower-layer convolutional layers, and gradually unfreeze the upper-layer network for training to avoid overfitting.
5. A method for detecting apple surface defects based on deep learning according to claim 1, characterized in that, In step S4, identifying the defect positions and types on the apple surface includes: During the defect recognition process, use bounding box regression or semantic segmentation to accurately locate the defect positions and mark the specific coordinates of the defects; And adopt multi-task learning to perform defect classification and localization simultaneously to improve the detection efficiency.
6. A method for detecting apple surface defects based on deep learning according to claim 1, characterized in that, Step S5 includes: Generate information including defect type, defect size, and defect area according to the recognition results and generate a quality score for each apple; And adopt a classification strategy based on decision trees or support vector machines to classify apples according to defect types and severity.
7. A method for detecting apple surface defects based on deep learning according to claim 1, characterized in that, It also includes: S6. Receive the recognition result of correcting errors in the defect report as the corrected training data, and combine it with new defect samples to retrain the apple surface defect detection model regularly.
8. An apple surface defect detection system based on deep learning, characterized in that, It includes: An acquisition and preprocessing module, which is used to acquire the image data of apple surface defects and perform preprocessing; The image data includes: images of five types, namely healthy, mechanical damage, pests and diseases, wrinkles, and rot, under various lighting conditions and different backgrounds; A feature extraction module, which is used to perform multi-scale feature extraction on the apple surface image by using an improved convolutional neural network combined with a deep residual network and an attention mechanism, and extract the apple surface defect features; A training module, which is used to extract the apple surface defect features; use transfer learning to fine-tune the pre-trained model, and train the apple surface defect detection model; An identification module, which is used to analyze the image data of the apple to be detected by using the trained apple surface defect detection model, and identify the defect position and type on the apple surface; A report generation module, which is used to generate a corresponding defect report according to the recognition result, and classify or remove the defective apples.
9. A device, characterized in that, It includes: At least one processor; And a memory communicatively connected to the at least one processor; Wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute any one of the apple surface defect detection methods based on deep learning as claimed in claims 1-7.
10. A storage medium, characterized in that, Instructions are stored in the storage medium, and when the instructions are run on the terminal, an apple surface defect detection method based on deep learning as claimed in any one of claims 1-7 can be realized.