Apple skin defect detection method and system based on deep learning
Through the improved YOLOv5s model, combined with the DSCS structure and WIoU loss function, the problem of insufficient detection accuracy of Apple's epidermal defects is solved, and more efficient detection of Apple's epidermal defects is achieved.
Patent Information
- Application Number
- CN202310888622.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-19
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2043-07-19
AI Technical Summary
In the prior art, the detection accuracy of Apple defects is insufficient, making it difficult to effectively detect the complex shape and color characteristics of Apple's skin.
Using the improved YOLOv5s model, the detection accuracy is improved by building a network including DSCS structure, Neck module and Head module, shallow and deep features are extracted and fused, and the WIoU loss function is used for high-quality annotation focus.
It significantly improves the accuracy of Apple's epidermal defect detection, enhances the network's feature information richness and small object detection capabilities, and reduces the influence of subjective factors.
Smart Images

Figure CN116843668B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of nondestructive testing of apple quality, and in particular to a method and system for detecting apple skin defects based on deep learning. Background Art
[0002] Apple is one of the most productive fruits in the world, and it is also the fruit with the largest planting area and the highest yield in China. According to statistics from the Foreign Agricultural Service of the United States Department of Agriculture, China's apple production accounts for more than 50% of the world's apple production all year round. With the development of science and technology and economy today, apple production efficiency has become the most important market competition factor, and the rapid production of high-quality agricultural products is the only way for the industry to win. However, most of the current agricultural production is still primitive manual production. Traditional manual planting, picking and sorting are inefficient and easily affected by other factors. In order to solve the corresponding problems and achieve efficient production of agricultural products, mechanization and intelligence are widely used in agriculture, gradually liberating agricultural productivity, which is the only way out for agriculture today.
[0003] The mechanization of agricultural products industry is mainly reflected in planting, picking and grading, and agricultural products grading is the main direction of mechanization in the apple industry. Efficiently screening out high-quality apples is a key step in brand competition. Since apples are easily damaged by collisions, the requirements for mechanical equipment design are high. At present, apple grading is mainly done manually. However, due to the existence of subjective factors, the grading accuracy is often reduced. It can be seen that automated, high-speed and efficient apple grading technology is favored by fruit farmers. More and more actual engineering projects are committed to building intelligent, reliable, fast and efficient apple grading systems.
[0004] At present, the mainstream non-destructive testing technologies used for apple quality inspection and grading are mainly machine vision and infrared spectroscopy technology. In recent years, technology has developed rapidly, and machine vision and deep learning have gradually become the mainstream methods in the current field of non-destructive testing. In the early days, due to insufficient computer computing power, it was often easy to be hindered in the practice process. With the development of computer technology, it has gradually been applied to all walks of life. Machine vision is an image processing technology that combines multidisciplinary technologies such as computer technology, electronic technology, artificial intelligence, and image processing. Images are collected through hardware devices, and the collected data is transmitted back to the computer for analysis and processing to achieve further application. The automated inspection of the external quality of apples mainly detects multiple indicators such as the diameter, color, and external defects of apples. Among them, the external defect detection has the characteristics of complex defect shape and color characteristics. Traditional methods often have insufficient detection accuracy, and it is necessary to strengthen the detection of apple defects. Summary of the invention
[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem of insufficient accuracy in apple defect detection in the prior art.
[0006] In order to solve the above technical problems, the present invention provides an apple skin defect detection method based on deep learning, comprising:
[0007] Get an apple skin image;
[0008] Build an improved YOLOv5s model;
[0009] Performing defect detection on the apple skin image by using the improved YOLOv5s model, and outputting an apple skin defect detection result;
[0010] Among them, the improved YOLOv5s model includes a Backbone module, a Neck module and a Head module connected in sequence, the Backbone module includes several DSCS structures, the DSCS structure is used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features; the Neck module is used to perform feature fusion on the features filtered by the Backbone module; the Head module is used to predict the features fused by the Neck module.
[0011] In one embodiment of the present invention, the Backbone module includes a first CBS unit, a second CBS unit, a first DSCS structure, a third CBS unit, a second DSCS structure, a fourth CBS unit, a third DSCS structure, a fifth CBS unit, a first C3 unit and a SPFF unit connected in sequence;
[0012] Among them, the first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, the first C3 unit is used for feature extraction, and the SPFF unit is used for feature fusion of feature maps of different scales.
[0013] In one embodiment of the present invention, the first DSCS structure, the second DSCS structure and the third DSCS structure all include a Dense module and a SE module connected in sequence, and the output of the Dense module and the output of the SE module are added and then connected to the tenth CBS unit;
[0014] The Dense module is used to extract shallow features and deep features, and splice the shallow features and the deep features to obtain spliced features;
[0015] The SE module is used to perform feature screening on the splicing features obtained by the Dense module;
[0016] The Dense module includes several Dense layers connected in sequence, the input of the current Dense layer is connected to the output of the current Dense layer, the input of the next Dense layer is connected to the output of the next Dense layer, and the input of the current Dense layer is also connected to the output of the next Dense layer;
[0017] Each of the Dense layers includes a first BN layer, a first Conv layer, and a ReLU layer connected in sequence.
[0018] In one embodiment of the present invention, the Neck module includes a sixth CBS unit, a first upsampling layer, a first Concat layer, a second C3 unit, a seventh CBS unit, a second upsampling layer, a second Concat layer, a third C3 unit, an eighth CBS unit, a third Concat layer, a fourth C3 unit, a ninth CBS unit, a fourth Concat layer and a fifth C3 unit connected in sequence;
[0019] The output of the SPFF unit is connected to the input of the sixth CBS unit, the sixth CBS unit is also connected to the fourth Concat layer, the third DSCS structure is also connected to the first Concat layer, the second DSCS structure is also connected to the second Concat layer, and the seventh CBS unit is also connected to the third Concat layer;
[0020] The sixth CBS unit, the seventh CBS unit, the eighth CBS unit and the ninth CBS unit are all used for feature extraction, and the second C3 unit, the third C3 unit, the fourth C3 unit and the fifth C3 unit are all used for feature extraction.
[0021] In one embodiment of the present invention, the Head module includes a first Decoupled Head unit, a second Decoupled Head unit and a third Decoupled Head unit;
[0022] Wherein, the first Decoupled Head unit is connected to the third C3 unit, the second Decoupled Head unit is connected to the fourth C3 unit, and the third Decoupled Head unit is connected to the fifth C3 unit;
[0023] The first Decoupled Head unit, the second Decoupled Head unit and the third Decoupled Head unit each include a first CBE unit and a second CBE unit connected in sequence, the second CBE unit is connected to a first branch and a second branch respectively, the first branch includes a third CBE unit, a fourth CBE unit and a Cls layer connected in sequence, the second branch includes a fifth CBE unit, the fifth CBE unit is connected to a third branch and a fourth branch respectively, the third branch includes a sixth CBE unit and a Reg layer connected in sequence, the fourth branch includes a seventh CBE unit and an Obj layer connected in sequence;
[0024] The first to seventh CBE units each include a second Conv layer, a second BN layer, and an ELU layer connected in sequence;
[0025] The Cls layer is used to output the category of the object, the Reg layer is used to output the coordinate information of the object, and the Obj layer is used to determine whether there is an object.
[0026] In one embodiment of the present invention, the first C3 unit to the fifth C3 unit each include a fifth branch and a sixth branch, the fifth branch includes a third Conv layer and a Bottleneck unit connected in sequence, the sixth branch includes a fourth Conv layer, the Bottleneck unit and the fourth Conv layer are both connected to a fifth Concat layer, and the fifth Concat layer is connected to the fifth Conv layer;
[0027] The Bottleneck unit includes a sixth Conv layer, a seventh Conv layer and an addition layer connected in sequence, and the input of the sixth Conv layer is also connected to the input of the addition layer;
[0028] The SPFF unit includes an eighth Conv layer, the eighth Conv layer is connected in parallel to the first maximum pooling layer, the second maximum pooling layer, the third maximum pooling layer and the sixth Concat layer, and the first maximum pooling layer, the second maximum pooling layer, and the third maximum pooling layer are all connected to the sixth Concat layer, and the sixth Concat layer is connected to the ninth Conv layer;
[0029] The first to ninth CBS units each include a tenth Conv layer, a third BN layer, and a SiLU layer connected in sequence.
[0030] In one embodiment of the present invention, the improved YOLOv5s model uses WIoU as the loss function during training, and the formula of WIoU is:
[0031] L WIoU =rR WIoUL IoU
[0032] Where r represents a constant factor, (x,y) represents the coordinates of the target prediction center point, (x gt ,y gt ) represents the coordinates of the center point of the target’s real position, W g Indicates the minimum enclosing rectangle width of the envelope prediction box and the real box, H g Indicates the minimum enclosing rectangle height of the envelope prediction box and the real box, δ represents the preset threshold, α represents the size of nonlinear capability mapping, L IoU Represents the IoU loss between the real box and the predicted box, Indicates the current object prediction IoU loss, Represents the mean IoU loss of all targets.
[0033] In order to solve the above technical problems, the present invention provides an apple skin defect detection system based on deep learning, comprising:
[0034] Acquisition unit: used to acquire apple skin images;
[0035] Construction unit: used to build an improved YOLOv5s model;
[0036] Detection unit: used to perform defect detection on the apple skin image by using the improved YOLOv5s model, and output the apple skin defect detection result;
[0037] Among them, the improved YOLOv5s model includes a Backbone module, a Neck module and a Head module connected in sequence, the Backbone module includes several DSCS structures, the DSCS structure is used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features; the Neck module is used to perform feature fusion on the features filtered by the Backbone module; the Head module is used to predict the features fused by the Neck module.
[0038] In order to solve the above technical problems, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the apple skin defect detection method based on deep learning are implemented.
[0039] In order to solve the above technical problems, the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned apple skin defect detection method based on deep learning are implemented.
[0040] The above technical solution of the present invention has the following advantages compared with the prior art:
[0041] The present invention improves the traditional YOLOv5s network. Specifically, the C3 module in the traditional YOLOv5s network is replaced by constructing a deep-shallow channel feature selection structure (i.e., DSCS structure). The improved YOLOv5s network can extract not only deep features but also shallow features through the Dense module, thereby improving the small target detection capability. The deep features and shallow features retained by the Dense module are screened through the SE module, thereby making the improved YOLOv5s network have richer feature information and greatly improving the network accuracy.
[0042] Aiming at the problem that coupling phenomenon is prone to occur when all regression tasks are completed at the same layer at the same time, the present invention adopts a decoupling head to realize decoupled regression. Finally, aiming at the quality problem of annotation, WIoU is used to focus on high-quality annotation. The three modules work together to improve the accuracy of the network. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.
[0044] Figure 1 is a flow chart of the method of the present invention;
[0045] Figure 2 It is a traditional YOLOv5s network structure diagram in an embodiment of the present invention;
[0046] Figure 3 is a structural diagram of a DSCS in an embodiment of the present invention;
[0047] Figure 4 It is a structural diagram of a Dense module in an embodiment of the present invention;
[0048] Figure 5 is a structural diagram of a decoupling head in an embodiment of the present invention;
[0049] Figure 6 : is a diagram of the improved YOLOv5s network structure in an embodiment of the present invention;
[0050] Figure 7 1 is a diagram of the nondestructive testing results of apples in an embodiment of the present invention. DETAILED DESCRIPTION
[0051] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0052] Embodiment 1
[0053] Reference Figure 1 As shown, the present invention relates to an apple skin defect detection method based on deep learning, comprising:
[0054] Get an apple skin image;
[0055] Build an improved YOLOv5s model;
[0056] Performing defect detection on the apple skin image by using the improved YOLOv5s model, and outputting an apple skin defect detection result;
[0057] Among them, the improved YOLOv5s model includes a Backbone module, a Neck module and a Head module connected in sequence. In this embodiment, the Backbone module includes several self-constructed DSCS structures. The DSCS structure is used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features; the Neck module is used to perform feature fusion on the features filtered by the Backbone module; the Head module is used to predict the features fused by the Neck module.
[0058] The following is a detailed introduction to this embodiment:
[0059] The main process of the YOLOv5s network includes multiple stages such as data labeling, data enhancement, Anchor regression, and model training. First, the data is labeled. Labelimg is used to label the data according to the specific type and location of the defect to obtain a labeling file. Then, offline data enhancement operations such as Gaussian blur, linear enhancement, and random cropping are performed on the image based on the image information and the corresponding annotation information. Then, the Mosica online data enhancement method is used before network training: four different random images are spliced at the four corners using a random center point, and affine transformation is performed according to the splicing position and scaling size, which expands the diversity of the data set. The Anchor regression is based on the target box size and shape of the current data set. Kmeans clustering is performed, and a group of the most suitable Anchor box sizes and shapes are clustered. This group of boxes has a certain adaptability to the shape and size of the target, which is conducive to the final model prediction.
[0060] The second is the model training stage. The model is trained for the positive sample matching strategy. The main difference in the model training stage lies in the difference in network structure and regression loss function. The network structure has certain requirements for the size and size of the target. For small target detection, a network that integrates more shallow information is required. For targets with complex shapes, a more complex network structure is required. The regression loss function plans the main learning direction of the network, and using different loss functions according to various situations in the training process can achieve better results.
[0061] See also Figure 2 The traditional YOLOv5s network structure is mainly divided into three parts: Backbone module, Neck module and Head module. The Backbone module is mainly composed of CBS unit and C3 unit. It obtains high-dimensional semantic information through continuous downsampling and feature extraction. However, CBS unit and C3 unit do not save the shallow information of the network. Continuous feature extraction and abstraction will lead to the loss of detail information and thus the loss of the ability to detect small targets. The Neck module mainly realizes the fusion processing of information, which fuses shallow information with deep information and inputs the fused features to the Head module for regression. Although the Neck module fuses shallow information, this fusion is limited and cannot solve the fundamental problem. Finally, there is the Head module. The Head module regresses the target box parameters according to the previous output to obtain the final prediction value, but the Head module only uses one network layer to regress all parameters, which will lead to coupling phenomenon.
[0062] In view of the problems existing in the YOLOv5s network, this embodiment proposes an improved YOLOv5s model. Figure 3 In the improved YOLOv5s model, this embodiment designs a deep and shallow feature selection structure (i.e., DSCS structure) based on the Dense module and the SE module. It filters the deep information and shallow information retained by the Dense module on the basis of the Dense module. The Dense module structure diagram is shown in Figure 4 As shown in the figure, the DSCS structure integrates deep information and shallow information, strengthens the important weights in the feature information, and weakens the influence of redundant weights, which plays a role in selecting shallow information and deep information. Compared with the C3 module in the traditional YOLOv5s network, it has richer feature information and greatly improves the network accuracy.
[0063] The traditional YOLOv5s network uses the CIoU loss function. The CIoU loss function introduces the aspect ratio factor based on D IoU, adds the aspect ratio of the target frame and the prediction frame into the network training target, and improves the network accuracy. This embodiment uses WIoU to replace the original CIoU loss, deals with the quality problem of the target frame, and weakens the influence of subjective factors. The WIoU calculation method is shown in formulas (1)-(4). Nonlinear gain is provided for losses of different outliers by setting parameters and controlling.
[0064] L WIoU =rR WIoU L IoU (1)
[0065]
[0066]
[0067]
[0068] Among them, r represents the constant factor, (x, y) represents the coordinates of the target prediction center point, (x gt ,y gt ) represents the coordinates of the center point of the target’s real position, W g Indicates the minimum enclosing rectangle width of the envelope prediction box and the real box, H g represents the minimum enclosing rectangle height of the envelope prediction box and the real box, δ represents the preset threshold, α represents the nonlinear capability mapping size, and L IoU Represents the IoU loss between the real box and the predicted box, Indicates the current object prediction IoU loss, Represents the mean IoU loss of all targets.
[0069] By calculating R WIoU Penalty term and L IoU The product of , and then add a factor r to add nonlinear gain to the annotation boxes of different outlier degrees. Where β is Divide by the mean By calculation With its mean The ratio of β is used as the outlier of the frame relative to all frames. When predicting all frames, the target frame with a larger β offset is considered to be a low-quality target frame. Then, according to the set threshold δ, a lower gain is provided for the target frame greater than the threshold, and a higher gain is provided for the target frame with a smaller β offset. This achieves nonlinear gain for high-quality target frames and low-quality target frames, thereby achieving dynamic focus on the target frame and improving the quality of network learning.
[0070] To sum up, the improved YOLOv5s network structure is as follows Figure 6As shown. The overall process of the improved YOLOv5 in this embodiment is: perform preliminary feature extraction on the input image through the DSCS module, while retaining the corresponding shallow information, fuse and select the shallow information and deep information, strengthen the learning of important features and weaken the influence of redundant features, and use the selected features for the Neck part input. Secondly, in view of the coupling phenomenon in the Head part, the decoupling prediction method is used to eliminate the coupling between parameters, and make full use of the shallow information fused by DSCS to realize the target frame regression. Finally, WIoU is used to make the network focus on the learning of high-quality target frames, which alleviates the labeling errors caused by subjective reasons to a certain extent and improves the overall network learning quality.
[0071] The following is a detailed introduction to the specific structure of the YOLOv5s model:
[0072] The improved YOLOv5s model of this embodiment includes a Backbone module, a Neck module and a Head module which are connected in sequence.
[0073] Furthermore, the Backbone module includes a first CBS unit, a second CBS unit, a first DSCS structure, a third CBS unit, a second DSCS structure, a fourth CBS unit, a third DSCS structure, a fifth CBS unit, a first C3 unit and an SPFF unit connected in sequence. Among them, the first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, mainly to adjust the size of the network output graph; the first DSCS structure, the second DSCS structure and the third DSCS structure are all used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features, the first C3 unit is used for feature extraction (but does not retain the input information, has a stronger feature extraction capability, and mines deeper features), and the SPFF unit is used to perform feature fusion of feature maps of different scales.
[0074] For further information, see Figure 3, the first DSCS structure, the second DSCS structure and the third DSCS structure all include a Dense module and a SE module (i.e., a channel attention module) connected in sequence. The output of the Dense module and the output of the SE module are added and connected to the tenth CBS unit. The role of the addition here is to strengthen the important weights in the feature information and weaken the influence of redundant weights. The Dense module is used to extract shallow features and deep features, and splice the shallow features and deep features to obtain spliced features; the SE module is used to perform feature screening on the spliced features obtained by the Dense module. The SE module can strengthen the channel features of the input feature map without changing the size of the input feature map; the Dense module includes several Dense layers connected in sequence, the input of the current Dense layer is connected to the output of the current Dense layer, the input of the next Dense layer is connected to the output of the next Dense layer, and the input of the current Dense layer is also connected to the output of the next Dense layer; each of the Dense layers includes the first BN layer, the first Conv layer and the ReLU layer connected in sequence. It should be noted that, Figure 4 Only two Dense layers are given as an illustration.
[0075] Furthermore, the Neck module includes a sixth CBS unit, a first upsampling layer, a first Concat layer, a second C3 unit, a seventh CBS unit, a second upsampling layer, a second Concat layer, a third C3 unit, an eighth CBS unit, a third Concat layer, a fourth C3 unit, a ninth CBS unit, a fourth Concat layer and a fifth C3 unit connected in sequence. The output of the SPFF unit is connected to the input of the sixth CBS unit, the sixth CBS unit is also connected to the fourth Concat layer, the third DSCS structure is also connected to the first Concat layer, the second DSCS structure is also connected to the second Concat layer, and the seventh CBS unit is also connected to the third Concat layer; the sixth CBS unit, the seventh CBS unit, the eighth CBS unit and the ninth CBS unit are all used for feature extraction and adjusting the size of the network output graph, and the second C3 unit, the third C3 unit, the fourth C3 unit and the fifth C3 unit are all used for feature extraction (but the input information is not retained, with stronger feature extraction capabilities and deeper features).
[0076] For further information, see Figure 5The Head module includes a first Decoupled Head unit, a second Decoupled Head unit and a third Decoupled Head unit (ie, three decoupled heads). Among them, the first Decoupled Head unit is connected to the third C3 unit, the second Decoupled Head unit is connected to the fourth C3 unit, and the third Decoupled Head unit is connected to the fifth C3 unit; the first Decoupled Head unit, the second Decoupled Head unit and the third Decoupled Head unit all include a first CBE unit and a second CBE unit connected in sequence, the second CBE unit is respectively connected to the first branch and the second branch, the first branch includes a third CBE unit, a fourth CBE unit and a Cls layer connected in sequence, the second branch includes a fifth CBE unit, the fifth CBE unit is respectively connected to the third branch and the fourth branch, the third branch includes a sixth CBE unit and a Reg layer connected in sequence, and the fourth branch includes a seventh CBE unit and an Obj layer connected in sequence; the first CBE unit to the seventh CBE unit all include a second Conv layer, a second BN layer and an ELU layer connected in sequence; the Cls layer is used to output the category of the object, the Reg layer is used to output the coordinate information (regression box information) of the object, and the Obj layer is used to determine whether there is an object.
[0077] Furthermore, the first C3 unit to the fifth C3 unit each include a fifth branch and a sixth branch, the fifth branch includes a third Conv layer and a Bottleneck unit connected in sequence, the sixth branch includes a fourth Conv layer, the Bottleneck unit and the fourth Conv layer are both connected to the fifth Concat layer, and the fifth Concat layer is connected to the fifth Conv layer.
[0078] Furthermore, the Bottleneck unit includes a sixth Conv layer, a seventh Conv layer and an addition layer (Add layer) connected in sequence, and the input of the sixth Conv layer is also connected to the input of the addition layer.
[0079] Further, the SPFF unit includes an eighth Conv layer, which is connected in parallel to the first maximum pooling layer, the second maximum pooling layer, the third maximum pooling layer and the sixth Concat layer, and the first maximum pooling layer, the second maximum pooling layer and the third maximum pooling layer are all connected to the sixth Concat layer, and the sixth Concat layer is connected to the ninth Conv layer.
[0080] Further, the first to ninth CBS units each include a tenth Conv layer, a third BN layer, and a SiLU layer connected in sequence.
[0081] Figure 7 In order to perform non-destructive testing on apples using the improved YOLOv5s model of this embodiment, it is not difficult to find that the actual testing effect is good.
[0082] Embodiment 2
[0083] This embodiment provides an apple skin defect detection system based on deep learning, including:
[0084] Acquisition unit: used to acquire apple skin images;
[0085] Construction unit: used to build an improved YOLOv5s model;
[0086] Detection unit: used to perform defect detection on the apple skin image by using the improved YOLOv5s model, and output the apple skin defect detection result;
[0087] Among them, the improved YOLOv5s model includes a Backbone module, a Neck module and a Head module connected in sequence, the Backbone module includes several DSCS structures, the DSCS structure is used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features; the Neck module is used to perform feature fusion on the features filtered by the Backbone module; the Head module is used to predict the features fused by the Neck module.
[0088] Embodiment 3
[0089] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the apple skin defect detection method based on deep learning described in Embodiment 1 are implemented.
[0090] Embodiment 4
[0091] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the apple skin defect detection method based on deep learning described in Embodiment 1 are implemented.
[0092] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiment, complete software embodiment, or the embodiment in combination with software and hardware. Moreover, the present application can adopt the form of the computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can adopt various computer languages to realize, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0093] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0096] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0097] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.
Claims
1. A method for detecting apple skin defects based on deep learning, characterized in that: include: Get an apple skin image; Build an improved YOLOv5s model; Performing defect detection on the apple skin image by using the improved YOLOv5s model, and outputting an apple skin defect detection result; The improved YOLOv5s model includes a Backbone module, a Neck module and a Head module connected in sequence, the Backbone module includes several DSCS structures, the DSCS structure is used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features; the Neck module is used to perform feature fusion on the features filtered by the Backbone module; the Head module is used to predict the features fused by the Neck module; The Backbone module includes a first CBS unit, a second CBS unit, a first DSCS structure, a third CBS unit, a second DSCS structure, a fourth CBS unit, a third DSCS structure, a fifth CBS unit, a first C3 unit and a SPFF unit connected in sequence; Among them, the first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, the first C3 unit is used for feature extraction, and the SPFF unit is used for feature fusion of feature maps of different scales; The first DSCS structure, the second DSCS structure and the third DSCS structure all include a Dense module and a SE module connected in sequence, and the output of the Dense module and the output of the SE module are added and then connected to the tenth CBS unit; The Dense module is used to extract shallow features and deep features, and splice the shallow features and the deep features to obtain spliced features; The SE module is used to perform feature screening on the splicing features obtained by the Dense module; The Dense module includes several Dense layers connected in sequence, the input of the current Dense layer is connected to the output of the current Dense layer, the input of the next Dense layer is connected to the output of the next Dense layer, and the input of the current Dense layer is also connected to the output of the next Dense layer; Each of the Dense layers includes a first BN layer, a first Conv layer, and a ReLU layer connected in sequence.
2. The apple skin defect detection method based on deep learning according to claim 1, characterized in that: The Neck module includes a sixth CBS unit, a first upsampling layer, a first Concat layer, a second C3 unit, a seventh CBS unit, a second upsampling layer, a second Concat layer, a third C3 unit, an eighth CBS unit, a third Concat layer, a fourth C3 unit, a ninth CBS unit, a fourth Concat layer and a fifth C3 unit connected in sequence; The output of the SPFF unit is connected to the input of the sixth CBS unit, the sixth CBS unit is also connected to the fourth Concat layer, the third DSCS structure is also connected to the first Concat layer, the second DSCS structure is also connected to the second Concat layer, and the seventh CBS unit is also connected to the third Concat layer; The sixth CBS unit, the seventh CBS unit, the eighth CBS unit and the ninth CBS unit are all used for feature extraction, and the second C3 unit, the third C3 unit, the fourth C3 unit and the fifth C3 unit are all used for feature extraction.
3. The apple skin defect detection method based on deep learning according to claim 2, characterized in that: The Head module includes a first Decoupled Head unit, a second Decoupled Head unit and a third Decoupled Head unit; Wherein, the first Decoupled Head unit is connected to the third C3 unit, the second Decoupled Head unit is connected to the fourth C3 unit, and the third Decoupled Head unit is connected to the fifth C3 unit; The first Decoupled Head unit, the second Decoupled Head unit and the third Decoupled Head unit each include a first CBE unit and a second CBE unit connected in sequence, the second CBE unit is connected to a first branch and a second branch respectively, the first branch includes a third CBE unit, a fourth CBE unit and a Cls layer connected in sequence, the second branch includes a fifth CBE unit, the fifth CBE unit is connected to a third branch and a fourth branch respectively, the third branch includes a sixth CBE unit and a Reg layer connected in sequence, the fourth branch includes a seventh CBE unit and an Obj layer connected in sequence; The first to seventh CBE units each include a second Conv layer, a second BN layer, and an ELU layer connected in sequence; The Cls layer is used to output the category of the object, the Reg layer is used to output the coordinate information of the object, and the Obj layer is used to determine whether there is an object.
4. The apple skin defect detection method based on deep learning according to claim 2, characterized in that: The first C3 unit to the fifth C3 unit all include a fifth branch and a sixth branch, the fifth branch includes a third Conv layer and a Bottleneck unit connected in sequence, the sixth branch includes a fourth Conv layer, the Bottleneck unit and the fourth Conv layer are both connected to a fifth Concat layer, and the fifth Concat layer is connected to the fifth Conv layer; The Bottleneck unit includes a sixth Conv layer, a seventh Conv layer and an addition layer connected in sequence, and the input of the sixth Conv layer is also connected to the input of the addition layer; The SPFF unit includes an eighth Conv layer, the eighth Conv layer is connected in parallel to the first maximum pooling layer, the second maximum pooling layer, the third maximum pooling layer and the sixth Concat layer, and the first maximum pooling layer, the second maximum pooling layer, and the third maximum pooling layer are all connected to the sixth Concat layer, and the sixth Concat layer is connected to the ninth Conv layer; The first to ninth CBS units each include a tenth Conv layer, a third BN layer, and a SiLU layer connected in sequence.
5. The apple skin defect detection method based on deep learning according to claim 1, characterized in that: The improved YOLOv5s model uses WIoU as the loss function during training, and the formula of WIoU is: L WIoU =rR WIoU L IoU Where r represents a constant factor, (x,y) represents the coordinates of the target prediction center point, (x gt ,y gt ) represents the coordinates of the center point of the target’s real position, W g Indicates the minimum enclosing rectangle width of the envelope prediction box and the real box, H g Indicates the minimum enclosing rectangle height of the envelope prediction box and the real box, δ represents the preset threshold, α represents the size of nonlinear capability mapping, L IoU Represents the IoU loss between the real box and the predicted box, Indicates the current object prediction IoU loss, Represents the mean IoU loss of all targets.
6. An apple skin defect detection system based on deep learning, characterized in that: include: Acquisition unit: used to acquire apple skin images; Construction unit: used to build an improved YOLOv5s model; Detection unit: used to perform defect detection on the apple skin image by using the improved YOLOv5s model, and output the apple skin defect detection result; The improved YOLOv5s model includes a Backbone module, a Neck module and a Head module connected in sequence, the Backbone module includes several DSCS structures, the DSCS structure is used to extract shallow features and deep features, and splice the shallow features and deep features, and then perform feature screening on the spliced features; the Neck module is used to perform feature fusion on the features filtered by the Backbone module; the Head module is used to predict the features fused by the Neck module; The Backbone module includes a first CBS unit, a second CBS unit, a first DSCS structure, a third CBS unit, a second DSCS structure, a fourth CBS unit, a third DSCS structure, a fifth CBS unit, a first C3 unit and a SPFF unit connected in sequence; Among them, the first CBS unit, the second CBS unit, the third CBS unit, the fourth CBS unit and the fifth CBS unit are all used for feature extraction, the first C3 unit is used for feature extraction, and the SPFF unit is used for feature fusion of feature maps of different scales; The first DSCS structure, the second DSCS structure and the third DSCS structure all include a Dense module and a SE module connected in sequence, and the output of the Dense module and the output of the SE module are added and then connected to the tenth CBS unit; The Dense module is used to extract shallow features and deep features, and splice the shallow features and the deep features to obtain spliced features; The SE module is used to perform feature screening on the splicing features obtained by the Dense module; The Dense module includes several Dense layers connected in sequence, the input of the current Dense layer is connected to the output of the current Dense layer, the input of the next Dense layer is connected to the output of the next Dense layer, and the input of the current Dense layer is also connected to the output of the next Dense layer; Each of the Dense layers includes a first BN layer, a first Conv layer, and a ReLU layer connected in sequence.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the apple skin defect detection method based on deep learning as described in any one of claims 1 to 5 are implemented.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the apple skin defect detection method based on deep learning as described in any one of claims 1 to 5 are implemented.
