Citrus maturity detection method based on improved YOLOv8 and related device

Through the improved YOLOv8 deep learning neural network model, the GhostConv and CARAFE operators and multi-dimensional collaborative attention mechanism are used to solve the problem of detection accuracy and speed balance in citrus maturity detection, and efficient detection under leaves occlusion is achieved, improving the efficiency of the detection under the condition of leaves Detection accuracy and efficiency.

CN118968502APending Publication Date: 2024-11-15WUYI UNIV
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411028096.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing citrus ripening method has the problem of balance of detection accuracy and speed. Especially when citrus fruits are blocked by leaves or the maturity is not uniform, the traditional method has poor detection effect and low manual identification and classification efficiency.

Method used

The improved YOLOv8 deep learning neural network model is adopted, combined with the GhostConv module and CARAFE upsampling operator, and a multi-dimensional collaborative attention mechanism is introduced to improve feature extraction capabilities and detection accuracy, so as to accurately detect citrus ripening under leaves occlusion.

Benefits of technology

It realizes fast and accurate citrus ripening, improves detection accuracy and efficiency, solves the problem of low detection efficiency in traditional methods, and is suitable for the identification and classification of Xinhui tangerines and other citrus varieties.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968502A_ABST
    Figure CN118968502A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a citrus maturity detection method based on improved YOLOv8 and a related device. The method comprises the following steps: acquiring a citrus image; the citrus image is input to a pre-trained deep learning neural network model for maturity detection to obtain a detection result, the deep learning neural network model takes YOLOv8 as a basic network, the network structure of the deep learning neural network model comprises a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, and the head network adopts a head network; the neck network adopts a GhostConv module to carry out convolution operation, the neck network adopts a CARAFE operator to carry out up-sampling operation, a multi-dimensional cooperative attention mechanism is introduced into the neck network, and the head network adopts a decoupling head structure. On the basis, the embodiment of the invention can be used for quickly and accurately detecting the maturity of the citrus under the condition that the citrus is shielded by the leaves.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of deep learning technology, and in particular to a citrus maturity detection method and related devices based on improved YOLOv8. Background Art

[0002] Xinhui dried tangerine peel is the best grade of Guangdong dried tangerine peel, a traditional Chinese medicinal material, the first of Guangdong's three treasures, and one of the top ten medicinal materials. Xinhui oranges are the raw materials of dried tangerine peel. At present, the picking of Xinhui oranges relies on manual labor. Since oranges of different maturity have different market values, it is necessary to regularly observe the maturity of citrus fruits, which greatly increases labor costs. At the same time, the determination of the maturity period of citrus fruits directly determines the transportation and storage methods of the fruits, affecting the production cost of the entire citrus. Since the fruits of Xinhui oranges are highly dense, the leaves are severely blocked, and the maturity is not uniform when the fruits are harvested, this problem becomes the most time-consuming and labor-intensive task. Therefore, the precise detection of the maturity of Xinhui oranges has important practical significance.

[0003] Traditional methods are limited by manpower and equipment costs, resulting in a lack of flexibility, efficiency and accuracy in citrus variety identification. The current fruit maturity target detection algorithm based on convolutional neural networks has a balance problem between detection accuracy and speed. Most algorithms with high detection accuracy have high model calculation parameters and slow detection speed. On the contrary, methods with low model calculation parameters and fast detection speed have low detection accuracy. At present, there are relatively few research methods on the detection of Xinhui orange maturity. During the growth and development process of Xinhui orange fruit, it is easily affected by factors such as light and environment, resulting in inconsistent fruit maturity and unclear differences in the maturity of adjacent fruits. These factors will affect the neural network model's ability to detect adjacent maturity and occluded fruits, resulting in poor detection results. Summary of the invention

[0004] The embodiment of the present invention provides a citrus maturity detection method and related devices based on improved YOLOv8, which can not only quickly and accurately detect Xinhui mandarin oranges, but also have a better detection effect on the maturity of citrus when it is blocked by leaves, while solving the problem of low efficiency of traditional manual recognition and classification.

[0005] In a first aspect, an embodiment of the present invention provides a citrus maturity detection method based on improved YOLOv8, comprising:

[0006] Get citrus images;

[0007] The citrus image is input into a pre-trained deep learning neural network model for maturity detection to obtain a detection result, wherein the deep learning neural network model is based on YOLOv8, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operation, the neck network adopts a CARAFE operator for upsampling operation, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure.

[0008] In some embodiments, the CARAFE operator includes a kernel prediction module and a feature-aware reconstruction module.

[0009] In some embodiments, the GhostConv module consists of a first part, a second part and a third part. The first part extracts features from an input feature map through a 1×1 convolution; the second part performs low-computational operations on a single channel through depthwise separable convolution; and the third part concatenates the feature map generated by the first part with the second redundant feature map to obtain an output feature map.

[0010] In some embodiments, the neck network further includes a downsampling layer, a feature concatenation layer, and multiple C2f modules, and the multi-dimensional collaborative attention mechanism is added after each of the C2f modules.

[0011] In some embodiments, the multi-dimensional collaborative attention mechanism consists of a squeezing conversion mechanism, an incentive conversion mechanism and aggregation. The squeezing conversion mechanism effectively combines features obtained through average pooling and standard deviation pooling; the incentive conversion mechanism is capable of adaptively determining the interaction coverage to obtain local feature interactions between channels; the multi-dimensional collaborative attention mechanism includes a top branch, a middle branch and a bottom branch, and the top branch, the middle branch and the bottom branch are aggregated together through operations in the integration stage to generate a refined feature map.

[0012] In some embodiments, the top branch is used to capture the interaction between features in the spatial dimension W, the middle branch is used to capture the interaction between features in the spatial dimension H, and the bottom branch is used to capture the interaction between channels.

[0013] In some embodiments, the head network uses two independent branches to perform an object classification task and a predicted bounding box regression task, the classification task uses a binary cross entropy loss function, and the predicted box boundary regression task uses a distribution focus loss function and a CIoU loss function.

[0014] In a second aspect, an embodiment of the present invention further provides a citrus maturity detection device based on improved YOLOv8, the device comprising:

[0015] An acquisition module, used for acquiring citrus images;

[0016] The detection module is used to input the citrus image into a pre-trained deep learning neural network model for maturity detection to obtain a detection result, wherein the deep learning neural network model is based on YOLOv8, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operation, the neck network adopts a CARAFE operator for upsampling operation, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting citrus maturity based on the improved YOLOv8 as described in the first aspect is implemented.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the citrus maturity detection method based on improved YOLOv8 as described in the first aspect.

[0019] According to the citrus maturity detection method and related device based on improved YOLOv8 provided by the embodiment of the present invention, the citrus maturity detection method based on improved YOLOv8 includes: obtaining a citrus image; inputting the citrus image into a pre-trained deep learning neural network model for maturity detection, and obtaining a detection result, wherein the deep learning neural network model is based on YOLOv8 as the basic network, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operation, the neck network adopts a CARAFE operator for upsampling operation, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure. Based on this, the embodiment of the present invention adopts a deep learning neural network model of the improved YOLOv8 algorithm, uses GhostConv to reduce the number of parameters and improve the detection accuracy, and combines the CARAFE upsampling operator and the multi-dimensional collaborative attention mechanism MCA to significantly improve the feature extraction capability. This deep learning neural network model can not only detect Xinhui oranges quickly and accurately, but also has better detection effects on citrus when they are obscured by leaves, while solving the problem of low efficiency of traditional manual identification and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1A is a flow chart of a citrus maturity detection method based on improved YOLOv8 provided by an embodiment of the present invention;

[0021] Figure 1B is a schematic diagram of the structure of a CARAFE operator provided by an embodiment of the present invention;

[0022] Figure 2 1 is a schematic diagram of the structure of a convolution and GhostConv module provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic diagram of the architecture of a multi-dimensional collaborative attention mechanism provided by an embodiment of the present invention;

[0024] Figure 4 It is a structural schematic diagram of a deep learning neural network model provided by an embodiment of the present invention;

[0025] Figure 5A This is a graph showing the results of maturity detection of immature Xinhui mandarins provided by one embodiment of the present invention;

[0026] Figure 5B This is a graph showing the maturity test results of semi-mature Xinhui mandarin oranges provided by one embodiment of the present invention;

[0027] Figure 5CThis is a graph showing the maturity detection results of mature Xinhui oranges provided by one embodiment of the present invention;

[0028] Fig. 6A This is an improved YOLOv8 detection effect diagram provided by an embodiment of the present invention;

[0029] Figure 6B This is a YOLOv8 detection effect diagram provided by an embodiment of the present invention;

[0030] Figure 6C This is an improved YOLOv7 detection effect diagram provided by an embodiment of the present invention;

[0031] Fig.6D This is an improved YOLOv9 detection effect diagram provided by an embodiment of the present invention;

[0032] Figure 7 is a schematic diagram of a citrus maturity detection device based on improved YOLOv8 provided by an embodiment of the present invention;

[0033] Figure 8 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the following drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0036] In the embodiments of the present invention, the words "further", "exemplarily" or "optionally" are used to indicate examples, illustrations or descriptions, and should not be interpreted as being more preferred or more advantageous than other embodiments or designs. The use of the words "further", "exemplarily" or "optionally" is intended to present related concepts in a specific way.

[0037] First, several terms involved in the present invention are analyzed:

[0038] YOLOv8: It is the next generation algorithm model developed by Ultra lytic ics after the YOLOv5 algorithm, which is used for object detection tasks.

[0039] Standard convolution (SC): Standard convolution is one of the most basic convolution operations in convolutional neural networks (CNNs). The basic idea is to use a fixed-size convolution kernel (also called a filter) to slide on the input feature map, multiply the local area of ​​the input feature map by the convolution kernel element by element, and then add the results to get the value of the corresponding position in the output feature map.

[0040] In order to more conveniently describe the working principle of the embodiment of the present invention later, an introduction to relevant technical scenarios is first given below.

[0041] Xinhui dried tangerine peel is the best grade of Guangdong dried tangerine peel, a traditional Chinese medicinal material, the first of Guangdong's three treasures, and one of the top ten medicinal materials. Xinhui oranges are the raw materials of dried tangerine peel. At present, the picking of Xinhui oranges relies on manual labor. Since oranges of different maturity have different market values, it is necessary to regularly observe the maturity of citrus fruits, which greatly increases labor costs. At the same time, the determination of the maturity period of citrus fruits directly determines the transportation and storage methods of the fruits, affecting the production cost of the entire citrus. Since the fruits of Xinhui oranges are highly dense, the leaves are severely blocked, and the maturity is not uniform when the fruits are harvested, this problem becomes the most time-consuming and labor-intensive task. Therefore, the precise detection of the maturity of Xinhui oranges has important practical significance.

[0042] Traditional methods are limited by manpower and equipment costs, resulting in a lack of flexibility, efficiency and accuracy in citrus variety identification. The current fruit maturity target detection algorithm based on convolutional neural networks has a balance problem between detection accuracy and speed. Most algorithms with high detection accuracy have high model calculation parameters and slow detection speed. On the contrary, methods with low model calculation parameters and fast detection speed have low detection accuracy. At present, there are relatively few research methods on the detection of Xinhui orange maturity. During the growth and development process of Xinhui orange fruit, it is easily affected by factors such as light and environment, resulting in inconsistent fruit maturity and unclear differences in the maturity of adjacent fruits. These factors will affect the neural network model's ability to detect adjacent maturity and occluded fruits, resulting in poor detection results.

[0043] Therefore, the existing technology has the following problems: (1) Target detection problem: How to improve the accuracy of target detection when the citrus fruit is not fully exposed, especially when it is blocked by leaves or the fruit overlaps. (2) Classification problem: How to improve the model to more accurately identify the maturity of citrus fruits when the maturity of fruits is inconsistent and the difference in maturity between adjacent fruits is not obvious.

[0044] Based on this, in order to balance the detection speed, detection accuracy and model calculation complexity, and better solve the detection problem of adjacent maturity, the present invention provides a citrus maturity detection method and related devices based on improved YOLOv8. Among them, the citrus maturity detection method based on improved YOLOv8 includes: obtaining a citrus image; inputting the citrus image into a pre-trained deep learning neural network model for maturity detection, and obtaining a detection result, wherein the deep learning neural network model is based on YOLOv8 as a basic network, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operation, the neck network adopts a CARAFE operator for upsampling operation, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure. Based on this, the embodiment of the present invention adopts a deep learning neural network model of the improved YOLOv8 algorithm, uses GhostConv to reduce the number of parameters and improve the detection accuracy, and combines the CARAFE upsampling operator and the multi-dimensional collaborative attention mechanism MCA to significantly improve the feature extraction capability. This deep learning neural network model can not only detect Xinhui oranges quickly and accurately, but also has better detection effects on citrus when they are obscured by leaves, while solving the problem of low efficiency of traditional manual identification and classification.

[0045] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0046] like Figure 1A As shown, Figure 1A It is a flowchart of a citrus maturity detection method based on improved YOLOv8 provided by an embodiment of the present invention. The citrus maturity detection method based on improved YOLOv8 may include but is not limited to steps S101 to S102.

[0047] Step S101: Acquire a citrus image;

[0048] Step S102: Input the citrus image into a pre-trained deep learning neural network model for maturity detection to obtain a detection result, wherein the deep learning neural network model is based on YOLOv8, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network. The backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operations, the neck network adopts a CARAFE operator for upsampling operations, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure.

[0049] It is understandable that the YOLO algorithm has significant advantages in speed and detection accuracy in the target detection network, and the YOLO series network has a wide range of applications in the agricultural field. In the actual citrus orchard scene, it is usually accompanied by complex environmental factors such as illumination, branch occlusion, and fruit overlap that affect detection. Therefore, the present invention aims to minimize the model calculation parameters while improving performance based on the YOLOv8 network model. To this end, GhostConv is used to replace some ordinary Convs to achieve a lighter network structure. At the same time, in order to solve the limitations of the original upsampling method, a lightweight upsampling operator CARAFE (Content-aware reassembly of features) is introduced to effectively restore the defect details of the citrus surface by predicting and reorganizing the input feature map. Finally, in order to further enhance the network's ability to extract citrus features, the MCA attention mechanism (Multi id imens iona l col aborative attent ion) is introduced in the nack part, which successfully suppresses the interference of non-target background information, thereby achieving more significant results in the citrus variety detection task.

[0050] It can be understood that the present invention takes YOLOv8 as the basic network, and the network structure of the deep learning neural network model can be divided into three parts: backbone network (Backbone), neck network (Neck) and head network (Head).

[0051] It can be understood that the backbone network is CSPDarknet53, whose main function is to extract features. Five features of different scales (B1-B5) are obtained by downsampling the input feature map five times. The C2f module uses gradient shunt connection to enrich the information flow of the feature extraction network and keep it lightweight. The CBS module in the backbone network performs batch normalization after convolution operation, and finally uses the SiLU activation function to obtain the output result. Finally, the backbone network uses the spatial pyramid pooling fast (SPPF) module to aggregate the feature map into a fixed-size feature map to achieve adaptive size output. SPPF reduces the amount of calculation and has lower latency by sequentially connecting three maximum pooling layers.

[0052] It is understandable that the main task of the neck network is to perform feature fusion, which is implemented using the PAN-FPN structure. The structure contains a series of convolutional layers and upsampling operations to gradually merge and restore high-resolution feature maps to the size of the original input image. In the neck convolution, GhostConv is used for convolution operations to effectively extract features and accelerate the training and reasoning process of the model while maintaining high detection accuracy. The upsampling operation uses a lightweight upsampling operator CARAFE to increase the spatial resolution of the feature map to improve the accuracy of target detection, especially for small targets. In addition, the neck network also includes a downsampling layer (Downsamp le), a feature concatenation layer (Concat), and a C2f module. An MCA attention mechanism is added after each C2f module to improve the feature extraction capability of the model and effectively integrate feature information at different levels. The purpose of these operations is to optimize the feature representation so that the network can better understand targets of different scales.

[0053] It is understandable that if Figure 1B As shown in Figure 1, CARAFE is a lightweight and efficient upsampling operator, which consists of a kernel prediction module and a content-aware reassembly module. It significantly improves the upsampling effect without introducing too many parameters and computational costs.

[0054] It is understandable that if Figure 2 As shown in the figure, GhostConv is a convolution operation in a convolutional neural network that can reduce the number of parameters while maintaining the expressiveness of the model. The GhostConv module consists of three parts: the first part extracts features from the input feature map through 1×1 convolution; the second part performs low-computation operations on single channels through depth-separable convolution; the third part concatenates the feature map generated by the first part with the second redundant feature map to obtain the output feature map.

[0055] It is understandable that if Figure 3As shown in the figure, the multidimensional collaborative attention mechanism (MCA) is a lightweight and efficient attention mechanism, which consists of squeeze, excitation and integration. The squeeze transformation mechanism obtains features through average-pooling and standard-difference pooling for effective combination. The excitation transformation mechanism can adaptively determine the interaction coverage to obtain local feature interactions between channels. Finally, the three branches are aggregated together through the operation in the integration stage and generate a refined feature map.

[0056] It is understandable that the head network is mainly responsible for outputting detection tasks. It adopts a decoupled head structure and uses two independent branches for object classification and predicted bounding box regression. The classification task uses binary cross entropy loss (BCE Loss), while the predicted box boundary regression task uses distribution focus loss (DFL) and CIoU. This structure improves detection accuracy and accelerates model convergence.

[0057] It is understandable that the present invention uses a pre-trained deep learning neural network model to detect the maturity of citrus images, which has the advantage that it can more efficiently process complex citrus image data and show higher accuracy in recognition accuracy. Using deep learning technology to detect the maturity of citrus can not only provide faster and more accurate test results, but also provide important technical support for ensuring the quality and authenticity of citrus, thereby improving the quality and efficiency of the production process.

[0058] It is understandable that the present invention adopts a deep learning neural network model of the improved YOLOv8 algorithm, uses GhostConv to reduce the number of parameters and improve detection accuracy, and combines the CARAFE upsampling operator and the multi-dimensional collaborative attention mechanism MCA to significantly improve the feature extraction capability. This deep learning neural network model can not only quickly and accurately detect Xinhui mandarin oranges, but also has a better detection effect on citrus when it is blocked by leaves, while solving the problem of low efficiency of traditional manual recognition and classification.

[0059] It can be understood that the embodiment of the present invention is based on target detection of citrus, which is suitable for the identification and classification of Xinhui oranges and other citrus varieties, and can be applied to intelligent fertilization in orchards, yield assessment, and automated citrus harvesting. The present invention provides technical support for the intelligent management of citrus orchards and the authenticity of Xinhui tangerine peel raw materials, and also has certain reference significance for subsequent work such as target detection, visual positioning, and classification and grading of other smart agricultural intelligent products.

[0060] In one embodiment, the CARAFE lightweight upsampling operator is used in Neck. When processing large-size images, the upsampling operation usually consumes huge computing resources. In addition, the upsampling operation of YOLOv8 may cause information loss, especially in the process of transposed convolution, some detail information will be lost, thereby affecting the model performance. In order to improve this problem, the present invention introduces a lightweight and efficient CARAFE operator in YOLOv8 to optimize the upsampling operation.

[0061] The CARAFE operator (Wang et al., 2019) consists of a kernel prediction module and a content-aware reassembly module. Its structure is as follows: Figure 1B Assume that the input feature map

[0062] X=H×W×C (1)

[0063] Where H is height, W is width and C is number of channels.

[0064] First, the compressed feature map is content encoded and the reconstructed kernel is generated. The upsampling kernel test module predicts the upsampling kernel at each position, and the feature reconstructed module completes the upsampling operation to obtain the output feature map x′, whose size is C×σH×σW, where σ is the upsampling rate. The upsampling kernel test module mainly completes content encoding, channel compression and upsampling normalization. In order to reduce the amount of calculation, the upsampling kernel is normalized to make the convolution kernel weight 1, and the number of channels is reduced to C. m . Assume that the upsampling kernel size is k m ×k m , for the compressed feature map, use k encoder ×k encoder The prediction upsampling kernel has the shape of The upsampling kernel w for each target position l′ l′ The upsampling kernel prediction module ψ is based on x l The neighborhood prediction of is shown in formula (2).

[0065] w l′ =ψ(N(xl ,k encoder )) (2)

[0066] In the feature reorganization module, each position x′ in the output feature map is mapped back to the input feature map x, with the target position as the center and the area size is k up ×k up The predicted upsampling kernel of the point is click-budgeted, and different channels at the same position share one upsampling kernel. The reorganization process is shown in formula (3), and the new feature map is finally obtained. is to convert x l The neighborhood and upsampling kernel w l′ The reorganized content perception module. After the introduction of the CARAFE operator, the receptive field can be effectively expanded, the features at all levels can be fully utilized, and the model accuracy can be improved.

[0067]

[0068] Where: N(x l ,k) represents the k×k area with l as the center x.

[0069] In one embodiment, the convolution part in Neck is improved. In the target detection task, the features extracted from the standard convolution have a repetitive process to a certain extent, and the continuous up-down sampling and dimension-up-down of feature maps of different sizes will cause a lot of computational consumption. Therefore, in response to this problem, a lightweight and efficient feature extraction module is designed in YOLOv8 using the GhostConv module. The GhostConv module can reduce the amount of computation and parameters of the network while maintaining the original channel size and the size of the convolution output feature map.

[0070] The GhostConv module consists of three parts: the input part extracts the input feature map through 1×1 convolution to obtain the feature map Y; the middle part performs low-computation operations on single channels through depth-wise convolution; the output part concatenates the feature map (blue) generated by the first part with the second redundant feature map (green) to obtain the output feature map. The feature map is n-dimensional, where Φ1…Φn represents feature maps of different dimensions, and Identity represents the identity mapping. h' is the output feature height; w^' is the output feature width; c is the number of input channels; g is the size of the conventional convolution kernel.

[0071] In ordinary convolution, the calculation formula is:

[0072] Flop conv =n×h′×w′×c×g×g (4)

[0073] Assuming the size of the linear transformation operation convolution kernel is r×r, each basic feature corresponds to s feature redundancy, and the number of s is less than the number of channels. The ordinary convolution method obtains m feature maps, and there is an identical transformation in the transformation process of the Ghost module. The calculation amount of using the Ghost module is:

[0074]

[0075] The computational cost of GhostConv compared with standard convolution is:

[0076]

[0077] Therefore, the computational complexity of Ghost convolution is about twice that of ordinary convolution. Using Ghost convolution instead of traditional convolution can improve the detection speed of the model while maintaining high detection accuracy and minimize the calculation parameters of the model.

[0078] In one embodiment, an MCA attention mechanism is introduced in Neck. Multidimensional collaborative attention (MCA) is a lightweight and efficient attention mechanism. In view of the problems existing in existing fruit detection methods, such as ignoring the modeling attention in the channel and spatial dimensions and the higher model complexity and computational complexity caused by the introduced methods, this paper introduces a lightweight and efficient method of multidimensional collaborative attention (MCA). The MCA module consists of Squeeze, Excitation and Integration, and its structure diagram is shown in the figure. Figure 3As shown in the figure. The squeeze transformation mechanism obtains effective combination of features through average-pooled and standard-deviation pooling. The excitation transformation mechanism is able to adaptively determine the interaction coverage to obtain local feature interactions between channels. Finally, the three branches are aggregated together through the operation in the integration stage and generate a refined feature map. This method uses a three-branch architecture and simultaneously infers channel, height, and dimensional attention without additional computational overhead. The top branch is used to capture the interaction between features in the spatial dimension W, the middle branch is used to capture the interaction between features in the spatial dimension H, and the bottom branch is responsible for capturing the interaction between channels. In the first two branches, permutation operations are used to capture the long-range dependencies between the channel dimension and any spatial dimension. Finally, in the integration stage, the outputs of all three branches are aggregated by a simple average.

[0079] like Figure 4 As shown, the present invention provides a deep learning neural network model for detecting the maturity of Xinhui mandarin oranges based on improved YOLOv8, and the model training includes the following steps:

[0080] 1. After configuring the training environment, download the pre-trained weights.

[0081] 2. Label Img software was used to annotate the Xinhui mandarin oranges of different maturity in the images. The annotation rules were as follows: (1) Annotate the fruits that occlude each other in the image but do not affect the manual judgment of maturity. (2) Do not annotate the fruits that are severely occluded. Among them, the maturity of the citrus dataset is divided into 579 fully ripe, 1069 unripe, and 527 medium ripe.

[0082] 3. When building the network detection model, set the training parameters, batch-size = 8, epochs = 300, and start training.

[0083] 4. Such as Figure 4 As shown in the figure, the main task of the neck network is to perform feature fusion, which is implemented using the PAN-FPN structure to process high-level features from the backbone network.

[0084] 5. Next comes a series of convolutional layers and upsampling operations, which aim to gradually merge and restore the high-resolution feature maps to the size of the original input image. These layers include GhostConv convolutional layers, CARAFE lightweight upsampling operators, downsamp le layers, and feature concatenation layers (Concat), which are used to effectively integrate feature information at different levels. Subsequently, the feature maps are further processed by the C2f module and the MCA attention mechanism, which significantly improves the feature extraction capability of the model.

[0085] 6. CARAFE (Content-aware reassembly of features) is a lightweight upsampling operator used to increase the spatial resolution of feature maps. It can better restore detail information by reassembling the input feature map in a content-aware manner, especially when processing small objects.

[0086] 7. Finally, the decoupled head structure is used in the detection part, using two independent branches for object classification and predicted bounding box regression. The final detection result is achieved by further processing the features.

[0087] 8. In order to improve the detection accuracy of the model, the input image is adjusted to 640×640 pixels. During the training process, the training process will only continue when the mAP value increases and the average loss decreases. In order to shorten the network training time and obtain the optimal training weights, Pat ience is set to 30, that is, in the early training stage, if the accuracy is not significantly improved, the training is stopped. The batch size is set to 16. The weight file is saved every 25 iterations, and the weight file with the highest mAP is used for detection. After training, the generated weights are used for detection, and the features are used to distinguish the maturity of citrus, and a result map with detection boxes and confidence levels is generated. The detection structure diagram of the maturity of Xinhui oranges is shown in the figure. Figure 5A , Figure 5B and Figure 5C As shown, Figure 5A The picture shows an unripe Xinhui mandarin orange. Figure 5B The picture shows a half-ripe Xinhui mandarin orange. Figure 5C Shown are ripe Xinhui mandarin oranges.

[0088] The image comparison of the test set is as follows FIG. 6A to FIG. 6D As shown, Fig. 6A , Figure 6B , Figure 6C , Fig.6DThese are the detection effect diagrams of improved YOLOv8, YOLOv8, YOLOv7, and YOLOv9. In the left figure, the improved YOLOv8 algorithm can detect the missed Xinhui oranges, while the YOLOv8 and YOLO9 algorithms have missed detections, and YOLOv7 has false detections. In the right figure, the improved YOLOv8 can complete the detection of the maturity of Xinhui oranges, while YOLOv7 and YOLOv9 have misjudged the maturity. In summary, in a complex natural environment full of leaf occlusion and uneven lighting, the improved YOLOv8 has advantages in detecting various maturity of Xinhui oranges and can complete the task of Xinhui orange maturity detection.

[0089] It can be understood that, in order to solve the problem of occlusion in citrus target detection, the present invention provides an algorithm for detecting the maturity of citrus and effectively detects citrus whose fruits are not fully exposed due to occlusion of leaves or overlapping of fruits, thereby improving the detection performance of the model.

[0090] It can be understood that for variety detection and maturity analysis of Xinhui oranges, the present invention extracts the attention mechanism of Xinhui orange characteristics, proposes a new neural network model combined with attention, and improves the feature extraction ability of the model, thereby realizing the ability of variety classification and maturity analysis and enriching the functions of the neural network model.

[0091] It can be understood that in order to address the problem that existing deep learning models are difficult to strike a balance between detection speed and accuracy, the present invention is based on a lightweight Xinhui citrus maturity detection algorithm, which reduces the number of model parameters and computational complexity by improving upsampling and convolution of the neck network while maintaining the original performance of the model.

[0092] It is understandable that when the Xinhui oranges are immature, their fruit color is similar to the background, both are green, and are easily confused with the background, resulting in missed detection. In this regard, the detection method of the present invention can balance the parameter quantity and accuracy, and can improve the detection performance of the model while reducing the parameter quantity. Therefore, the improved lightweight model of the present invention is easier to deploy in mobile devices, thereby promoting the development of smart agriculture.

[0093] In addition, if Figure 7 As shown, one embodiment of the present invention further discloses a citrus maturity detection device based on improved YOLOv8, the device comprising:

[0094] An acquisition module 110 is used to acquire a citrus image;

[0095] The detection module 120 is used to input the citrus image into a pre-trained deep learning neural network model for maturity detection to obtain a detection result, wherein the deep learning neural network model is based on YOLOv8, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network. The backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operations, the neck network adopts a CARAFE operator for upsampling operations, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure.

[0096] The citrus maturity detection device based on improved YOLOv8 in an embodiment of the present invention is used to execute the citrus maturity detection method based on improved YOLOv8 in the above embodiment. Its specific processing process is the same as the citrus maturity detection method based on improved YOLOv8 in the above embodiment, and will not be repeated here.

[0097] In addition, if Figure 8 As shown, an embodiment of the present invention also discloses an electronic device, including: at least one processor 210; at least one memory 220, for storing at least one program; when the at least one program is executed by the at least one processor 210, a citrus maturity detection method based on improved YOLOv8 as in any of the previous embodiments is implemented.

[0098] In addition, an embodiment of the present invention further discloses a computer-readable storage medium storing computer-executable instructions for executing a citrus maturity detection method based on improved YOLOv8 as in any of the previous embodiments.

[0099] The system architecture and application scenarios described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are equally applicable to similar technical problems.

[0100] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0101] In hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0102] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process or an execution thread, and a component may be located on a computer or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may communicate, for example, through local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems through signals).

Claims

1. A citrus maturity detection method based on improved YOLOv8, include: Get citrus images; The citrus image is input into a pre-trained deep learning neural network model for maturity detection to obtain a detection result, wherein the deep learning neural network model is based on YOLOv8, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operation, the neck network adopts a CARAFE operator for upsampling operation, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure.

2. The method according to claim 1, It is characterized in that The CARAFE operator includes a kernel prediction module and a feature perception and reconstruction module.

3. The method according to claim 1, It is characterized in that The GhostConv module consists of a first part, a second part and a third part. The first part extracts features from an input feature map through a 1×1 convolution; the second part performs low-computation operations on single channels through depthwise separable convolution; and the third part concatenates the feature map generated by the first part with the second redundant feature map to obtain an output feature map.

4. The method according to claim 1, It is characterized in that The neck network also includes a downsampling layer, a feature concatenation layer and multiple C2f modules, and the multi-dimensional collaborative attention mechanism is added after each C2f module.

5. The method according to claim 1, It is characterized in that The multi-dimensional collaborative attention mechanism consists of a squeeze conversion mechanism, an incentive conversion mechanism and aggregation. The squeeze conversion mechanism is to effectively combine features obtained by average pooling and standard deviation pooling; the incentive conversion mechanism is to adaptively determine the interaction coverage to obtain local feature interactions between channels; The multi-dimensional collaborative attention mechanism includes a top branch, a middle branch and a bottom branch, which are aggregated together through operations in an integration phase and generate a refined feature map.

6. The method according to claim 5, It is characterized in that The top branch is used to capture the interaction between features in the spatial dimension W, the middle branch is used to capture the interaction between features in the spatial dimension H, and the bottom branch is used to capture the interaction between channels.

7. The method according to claim 1, It is characterized in that The head network uses two independent branches to perform object classification tasks and predicted bounding box regression tasks. The classification task uses a binary cross entropy loss function, and the predicted box boundary regression task uses a distribution focus loss function and a CIoU loss function.

8. A citrus maturity detection device based on improved YOLOv8, It is characterized in that The device comprises: An acquisition module, used for acquiring citrus images; The detection module is used to input the citrus image into a pre-trained deep learning neural network model for maturity detection to obtain a detection result, wherein the deep learning neural network model is based on YOLOv8, and the network structure of the deep learning neural network model includes a backbone network, a neck network and a head network, the backbone network is CSPDarknet53, the neck network adopts a PAN-FPN structure, the neck network adopts a GhostConv module for convolution operation, the neck network adopts a CARAFE operator for upsampling operation, the neck network introduces a multi-dimensional collaborative attention mechanism, and the head network adopts a decoupled head structure.

9. An electronic device, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the citrus maturity detection method based on the improved YOLOv8 as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the citrus maturity detection method based on improved YOLOv8 as described in any one of claims 1 to 7.

Citation Information

Cited By

  • Pineapple maturity target detection method based on YOLOv10 model

    CN120088774A

  • Strawberry fruit detection system and method based on improved YOLOv8n network

    CN120299034A

  • Orchard citrus detection method based on improved YOLOv10n

    CN120931895A

  • Flame detection method for intelligent operation and maintenance fire-fighting inspection robot

    CN121582548A

  • A flame detection method for an intelligent operation and maintenance fire patrol robot

    CN121582548B