Litchi image classification method based on YOLO11 model and processing terminal

Abstract

The invention discloses a lychee image classification method based on a YOLO11 model, and the method comprises the steps: employing a lychee image database to train the YOLO11 model, and the YOLO11 model comprises a backbone network, a neck network, and a decision network; wherein the backbone network comprises a convolutional layer and a HaloAttention module, and the convolutional layer comprises a lightweight adaptive extraction convolution (LAE); the neck network comprises a CARAFE feature sampling block; and carrying out litchi variety prediction by adopting the trained YOLO11 model. According to the method, the local attention HaloAttention is introduced into the backbone network of the YOLO11 model, so that the feature expression capability in a local range is improved; the Conv convolution is replaced with lightweight adaptive extraction convolution LAE, and feature complementation between channels is enhanced; a CARAFE feature sampling block is introduced, and the sampling process is dynamically adjusted according to input features; the method can accurately predict the variety of the litchi fruit, and has the advantages of rapid identification, high identification accuracy and strong stability.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and classification, and in particular to a litchi image classification method and a processing terminal based on a YOLO11 model. Background Art

[0002] As a globally important tropical cash crop, lychee variety identification plays a key role in the fruit industry chain. Traditional classification methods based on manual phenotypic observations suffer from significant limitations, such as subjectivity and inefficiency, making them unable to meet the modern fruit market's demand for real-time, precise variety identification.

[0003] In recent years, deep learning technologies, particularly object detection algorithms, have made significant progress in image recognition. YOLO (You Only Look Once), as an efficient object detection algorithm, has become a mainstream choice for object detection and classification tasks due to its real-time performance and accuracy. YOLO11, the latest version of the YOLO series, further optimizes the network structure and detection accuracy, enabling accurate object detection and classification in more complex environments.

[0004] The task of identifying litchi varieties presents challenges such as high loss of effective features due to feature occlusion caused by dense fruit clusters, limited phenotypic variation between different litchi varieties (e.g., husk texture and fruit curvature), and color variations in the peel caused by illumination within the same variety. The existing YOLO11 model cannot effectively capture these subtle differences, making it difficult to achieve ideal results. In particular, the nearly circular shape of the litchi fruit is prone to projected distortion, and detailed features such as the distribution of thorns are easily lost in low-resolution images, resulting in a significant decrease in variety identification accuracy. Summary of the Invention

[0005] The present invention aims to at least partially address one of the problems in the related art. To this end, the present invention provides a litchi image classification method based on the YOLO11 model, which can quickly and accurately predict litchi varieties, with the advantages of rapid recognition, high recognition accuracy, and strong stability.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] A litchi image classification method based on the YOLO11 model, comprising:

[0008] The YOLO11 model is trained using the litchi image database, wherein the YOLO11 model includes a backbone network, a neck network, and a decision network; wherein the backbone network includes a convolutional layer and a local attention module HaloAttention, the convolutional layer includes a lightweight adaptive extraction convolution LAE; the neck network includes a CARAFE feature sampling block;

[0009] The trained YOLO11 model is used to predict litchi varieties.

[0010] Furthermore, the processing process of the local attention module HaloAttention on the litchi image includes:

[0011] The litchi image is segmented into several small blocks, and the boundary of each small block is expanded using the haloing operation to form a neighborhood window with additional boundary information;

[0012] Take each small block in the neighborhood window as a query, perform attention calculation with the neighborhood window, and extract context features from it;

[0013] Output the result of each small block to form an enhanced small block.

[0014] Furthermore, the method of introducing the local attention module HaloAttention into the YOLO11 model includes:

[0015] Construct the local attention module HaloAttention and add the local attention module HaloAttention to the backbone network of the YOLO11 model.

[0016] Furthermore, the method of introducing lightweight adaptive extraction convolution LAE into the YOLO11 model includes: creating a new python file, writing relevant code for lightweight adaptive extraction convolution LAE, adding lightweight adaptive extraction convolution LAE to the relevant task.py file, and modifying the corresponding parameters, and adding the constructed lightweight adaptive extraction convolution LAE to the second, fourth, sixth, and eighth layers of the backbone network respectively.

[0017] Furthermore, the processing process of the lightweight adaptive extraction convolution LAE on the litchi image includes:

[0018] The litchi image undergoes group convolution, which divides the input channels into multiple groups for convolution operation. The convolved features are rearranged to obtain the feature map.

[0019] The feature map undergoes adaptive average pooling to aggregate feature information in height and width dimensions;

[0020] The feature map after adaptive average pooling is normalized.

[0021] Furthermore, the processing of the litchi image by the CARAFE feature sampling block includes:

[0022] The input feature map is channel-compressed to generate a low-dimensional representation; the compressed features are encoded through a content encoder to generate the corresponding dynamic kernel; the generated dynamic kernel is normalized;

[0023] The input feature map interacts with the dynamic kernel to form an updated feature map.

[0024] Furthermore, the formation of the litchi image database includes:

[0025] Collect images of different varieties of litchi and label them by category;

[0026] The litchi images were preprocessed and a litchi image database was established.

[0027] Furthermore, the litchi image is preprocessed, specifically including:

[0028] Remove the litchi images that do not meet the standards in terms of angle, size, color and integrity;

[0029] Rotate, scale, and crop litchi images to expand the dataset size;

[0030] The processed litchi images are divided into training sets and test sets according to a preset ratio to establish a litchi image database.

[0031] Furthermore, the YOLO11 model is trained using the litchi image database, specifically including:

[0032] Use the training set to train the model and monitor the accuracy and TOP-1 curve of the YOLO11 model in real time during the training process;

[0033] Use the test set to evaluate the performance of the YOLO11 model, adjust the model parameters based on the evaluation results, and improve the model performance by optimizing the learning rate and batch size of the training process;

[0034] After multiple iterations and tuning, a trained YOLO11 model is obtained.

[0035] The present application also provides a processing terminal, including a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the processing terminal executes the litchi image classification method based on the YOLO11 model as described above.

[0036] The above technical solution provided by the embodiment of the present application has the following advantages over the existing technology: the present application introduces the local attention module HaloAttention (HaloAttention) into the backbone network (background) of the YOLO11 model to improve the feature expression capability within the local range; replaces the Conv convolution with the lightweight adaptive extraction convolution LAELAE, which reduces the computational cost and strengthens the feature complementarity between channels; introduces the CARAFE feature sampling block to dynamically adjust the sampling process according to the input features; uses the litchi image database to train and test the improved YOLO11 model to obtain a trained YOLO11 model; compared with the existing technology, the present invention adopts a litchi image classification method based on the YOLO11 model, which can quickly and accurately predict the variety of litchi, and has the advantages of fast recognition, high recognition accuracy and strong stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] In the attached figure:

[0040] Figure 1 1 is a flow chart of a litchi image classification method based on the YOLO11 model of the present invention;

[0041] Figure 2 2 is a block diagram of the structure of the local attention module HaloAttention shown in a specific embodiment of the present invention;

[0042] Figure 3 1 is a diagram of the YOLO11 model backbone network architecture according to a specific embodiment of the present invention;

[0043] Figure 4 This is an example diagram of the Top-1 accuracy rate shown in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific embodiments of the present invention are now described in detail with reference to the accompanying drawings. In the following description, it should be understood that the directions or positional relationships indicated by "front", "back", "up", "down", "left", "right", "longitudinal", "horizontal", "vertical", "horizontal", "top", "bottom", "inside", "outside", "head", "tail", etc. are based on the directions or positional relationships shown in the accompanying drawings and are constructed and operated in specific directions. They are only for the convenience of describing the technical solution and do not indicate that the mechanisms or components referred to must have specific directions. Therefore, they should not be understood as limiting the present invention.

[0045] It should also be noted that, unless otherwise clearly specified and limited, terms such as "installed", "connected", "connected", "fixed", and "set" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integrated connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. When an element is referred to as being "on" or "under" another element, the element can be "directly" or "indirectly" located on the other element, or there may be one or more intervening elements. The terms "first", "second", "third", etc. are only for the convenience of describing the present technical solution, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of such features. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0046] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, mechanisms, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0047] like Figure 1 As shown, the present application provides a litchi image classification method based on the YOLO11 model, including:

[0048] S1: The YOLO11 model is trained using the Litchi image database, wherein the YOLO11 model includes a backbone network, a neck network, and a decision network; wherein the backbone network includes a convolutional layer and a local attention module HaloAttention, the convolutional layer includes a lightweight adaptive extraction convolution LAE; the neck network includes a CARAFE feature sampling block.

[0049] Specifically include:

[0050] S11: Establishing a litchi image database; including:

[0051] S111: Collect images of different varieties of litchi and label them by category; the details are as follows:

[0052] In this example, the lychee images were primarily sourced from the Lychee Expo Center in Conghua District, Guangzhou, Guangdong Province. The image collection device was the rear camera of a Xiaomi 10 mobile phone, with an image resolution of 4032 × 3024 pixels. Shooting conditions included both sunny and cloudy days, and the images covered ten different lychee varieties: Baitangpeng, Feizixiao, Guanyinlu, Guiwei, Heiye, Lizhiwang, Huaizhi, Nuomici, Xianjinfeng, and Midingxiang. During the collection process, various lighting conditions and shooting angles were selected to increase the diversity of the target data. A total of 8,220 images were captured, with approximately 800 images of each variety.

[0053] S112: Preprocessing the litchi image to establish a litchi image database. Preprocessing the litchi image specifically includes:

[0054] Remove the litchi images that do not meet the standards in terms of angle, size, color and integrity;

[0055] Rotate, scale, and crop qualified litchi images to expand the dataset size;

[0056] Write a random allocation Python code to divide the processed litchi images into the train and val folders according to the preset ratio of 8:2, obtain the training set and test set, and establish a litchi image database.

[0057] S12: Build the YOLO11 model and improve the YOLO11 model, including:

[0058] Introduce the local attention module HaloAttention in the backbone of the YOLO11 module:

[0059] S121: Five models of the YOLO11 module (YOLO11n-cls, YOLO11s-cls, YOLO11m-cls, YOLO11l-cls, and YOLO11x-cls) were trained on the Lizhi image database. The experimental parameters were set as follows: 640×640 pixels were selected as the input image size of the model, SGD was selected as the model optimizer, the initial optimizer learning rate was set to 0.0001, the model was trained 100 times, and the batch size was 16. The recognition accuracy, recognition speed, and model size of the experiment were compared. YOLO11n-cls was selected as the base model and improved on it.

[0060] Create a new Python file and write the relevant code for the local attention module HaloAttention. Add the local attention module HaloAttention to the relevant task.py file and modify the relevant parameters. Add the constructed local attention module HaloAttention to the tenth layer of the backbone network of the YOLO11 model, and set the backbone related parameters to [-1, 1, HaloAttention, [2, 1]].

[0061] In this example, the structural diagram of the local attention module HaloAttention is as follows Figure 2 As shown, the processing process specifically includes:

[0062] The input image, i.e., the litchi image, is divided into multiple small blocks. After the image is divided into blocks, the boundary of each small block is extended using the haloing operation. Each small block has a 1-pixel boundary on its original area, forming a neighborhood window with additional boundary information. After haloing, each small block has information about the neighboring area, that is, the expanded neighborhood window contains some information from the surrounding small blocks;

[0063] Each small block in the neighborhood window is used as a query (Query), and attention calculation is performed with its expanded neighborhood window to extract important contextual features. The introduction of the attention mechanism enables each small block to not only learn its own features, but also obtain relevant contextual information from surrounding blocks, thereby enhancing feature expression;

[0064] The results of each small block are weightedly output by the attention mechanism to form a new feature map. The size of the output feature map is still the same as before block division, but the features within each block have been contextually enhanced and fused.

[0065] S122: Replace Conv convolution with lightweight adaptive extraction convolution LAELAE, as follows:

[0066] Create a new Python file and write the relevant code for the lightweight adaptive extraction convolution LAE. Add the lightweight adaptive extraction convolution LAE to the relevant task.py file and modify the corresponding parameters. Add the constructed lightweight adaptive extraction convolution LAE to the second, fourth, sixth, and eighth layers of the backbone respectively.

[0067] In this example, the image processing process of the lightweight adaptive extraction convolution LAELAE specifically includes:

[0068] The input feature map undergoes group convolution (groups = c / 16), which divides the input channels into multiple groups for convolution operation. The convolved features undergo a rearrangement operation (R), which rearranges the results of the group convolution into different spatial dimensions for feature representation.

[0069] The feature map undergoes adaptive average pooling (AP) to aggregate feature information in the height and width dimensions. The pooled feature map is further compressed and the number of channels is adjusted through a 1x1 convolution operation, which can learn important features between channels with less computational effort.

[0070] The feature map undergoes rearrangement (R) and Softmax normalization (S), so that the weight of each channel can be adaptively adjusted, further improving the network's expressive power.

[0071] The results of lightweight extraction and adaptive extraction are fused through multiplication, that is, the feature map of lightweight extraction is adjusted using the weight of adaptive extraction, and the fused feature map is added (Sum) to the input feature. The residual connection is used to ensure smooth information flow, and the final feature map of size is output.

[0072] S122: Introduce the CARAFE feature sampling block and dynamically adjust the sampling process according to the input features. The sampling process of the CARAFE feature sampling block is as follows:

[0073] The upsampling operation of the feature map is regarded as a process of feature reorganization. For each pixel in the output feature map, we can find its corresponding position L in the input feature map. The value of this point can be expressed as the dot product (weighted sum) of the pixels in a neighborhood centered on L in the input feature map and an upsampling kernel. Taking bilinear upsampling as an example, each pixel in the output feature map can be regarded as the dot product of a 2×2 upsampling kernel and a 2×2 neighborhood in the input feature map.

[0074] Specifically, the input feature map X is channel-compressed to generate a low-dimensional representation. This compression operation reduces the amount of computation and retains key information. The compressed features are then encoded through a content encoder to generate the corresponding dynamic kernel. The generated dynamic kernel is normalized to ensure consistent behavior at different positions to avoid deviations.

[0075] The input feature map X interacts with the previously predicted dynamic kernel to generate a convolution result in the local area. For each input feature, a new recombined feature is generated. The Reassembly operation combines information from different locations and performs feature fusion through a specific kernel to generate a new feature map with richer local and global features.

[0076] The backbone network architecture of the optimized YOLO11 model is as follows Figure 3 shown.

[0077] S13 uses the Litchi image database to train the YOLO11 model, specifically including:

[0078] S131: Use the training set of the Litchi dataset for model training. The parameters are as follows: select 640×640 pixels as the input model image size, select SGD as the model optimizer, set the initial optimizer learning rate to 0.0001, set the model training epochs to 100, and set the batch size to 16. Monitor the accuracy and top-1 curve during training to ensure the stability and effectiveness of the training process.

[0079] S132: Use the test set to evaluate the performance of the YOLO11 model, and adjust the model parameters based on the evaluation results, and improve the model performance by optimizing the learning rate and batch size of the training process;

[0080] S133: Through multiple iterations and optimization, the network hyperparameters and structure are adjusted, the number of network layers is modified, and the activation function is optimized to obtain the optimal litchi variety prediction model.

[0081] This application provides an example of the Top-1 accuracy of the litchi image classification method based on the YOLO11 model: Figure 4 Figure 2 shows the Top-1 accuracy trend of the YOLO11 model on the training set. The figure shows significant accuracy fluctuations in the early stages of training (approximately the first 10 iterations). As training progresses, the learning curve gradually stabilizes. Specifically, the model converges after 50 iterations. After 100 iterations, the classification accuracy stabilizes at a high level of 99%, demonstrating excellent generalization performance.

[0082] S2: Use the trained YOLO11 model to predict litchi varieties.

[0083] Input the collected litchi images into the trained YOLO11 model to obtain the litchi varieties.

[0084] The above-mentioned litchi image classification method based on improved YOLO11 and CARAFE feature sampling collects a litchi image database; preprocesses the litchi image database; builds a convolutional neural network based on the YOLO11 module, introduces the local attention module HaloAttention in the background of the YOLO11 module to improve the feature expression ability within the local range; replaces the Conv convolution with the lightweight adaptive extraction convolution LAELAE, which reduces the computational cost and strengthens the feature complementarity between channels; introduces the CARAFE feature sampling block to dynamically adjust the sampling process according to the input features; uses the litchi image database to train and test the improved YOLO11 model to obtain a trained litchi variety prediction model; compared with the existing technology, the present invention adopts the litchi image classification method based on improved YOLO11 and CARAFE feature sampling, which can quickly and accurately predict the litchi variety, and has the advantages of fast recognition, high recognition accuracy and strong stability.

[0085] The present application also provides a processing terminal, including a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the processing terminal implements the above-mentioned litchi image classification method based on the YOLO11 model.

[0086] It is understandable that the above embodiments only express the preferred implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the patent scope of the present invention. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present invention, the above technical features can be freely combined, and several deformations and improvements can be made, all of which fall within the scope of protection of the present invention. Therefore, all equivalent changes and modifications made to the scope of the claims of the present invention should fall within the scope of coverage of the claims of the present invention.

Claims

1. A litchi image classification method based on the YOLO11 model, characterized in that: include: The YOLO11 model is trained using the litchi image database, wherein the YOLO11 model includes a backbone network, a neck network, and a decision network; wherein the backbone network includes a convolutional layer and a local attention module HaloAttention, the convolutional layer includes a lightweight adaptive extraction convolution LAE; the neck network includes a CARAFE feature sampling block; The trained YOLO11 model is used to predict litchi varieties.

2. A litchi image classification method based on the YOLO11 model according to claim 1, characterized in that, The processing process of the local attention module HaloAttention on the litchi image includes: The litchi image is segmented into several small blocks, and the boundary of each small block is expanded using the haloing operation to form a neighborhood window with additional boundary information; Take each small block in the neighborhood window as a query, perform attention calculation with the neighborhood window, and extract context features from it; Output the result of each small block to form an enhanced small block.

3. A litchi image classification method based on the YOLO11 model according to claim 1, characterized in that, The methods of introducing the local attention module HaloAttention into the YOLO11 model include: Construct the local attention module HaloAttention and add the local attention module HaloAttention to the backbone network of the YOLO11 model.

4. A litchi image classification method based on the YOLO11 model according to claim 1, characterized in that, The method of introducing lightweight adaptive extraction convolution LAE into the YOLO11 model includes: creating a new python file, writing the relevant code of lightweight adaptive extraction convolution LAE, adding lightweight adaptive extraction convolution LAE to the relevant task.py file, and modifying the corresponding parameters, and adding the constructed lightweight adaptive extraction convolution LAE to the second, fourth, sixth, and eighth layers of the backbone network respectively.

5. A litchi image classification method based on the YOLO11 model according to claim 1, characterized in that, The processing process of the lightweight adaptive extraction convolution LAE on the litchi image includes: The litchi image undergoes group convolution, which divides the input channels into multiple groups for convolution operation. The convolved features are rearranged to obtain the feature map. The feature map undergoes adaptive average pooling to aggregate feature information in height and width dimensions; The feature map after adaptive average pooling is normalized.

6. A litchi image classification method based on the YOLO11 model according to claim 1, characterized in that, The processing process of the litchi image by the CARAFE feature sampling block includes: The input feature map is channel-compressed to generate a low-dimensional representation; the compressed features are encoded through a content encoder to generate the corresponding dynamic kernel; the generated dynamic kernel is normalized; The input feature map interacts with the dynamic kernel to form an updated feature map.

7. A litchi image classification method based on the YOLO11 model according to claim 1, characterized in that, The formation of the litchi image database includes: Collect images of different varieties of litchi and label them by category; The litchi images were preprocessed and a litchi image database was established.

8. A litchi image classification method based on the YOLO11 model according to claim 7, characterized in that, Preprocess the litchi image, including: Remove the litchi images that do not meet the standards in terms of angle, size, color and integrity; Rotate, scale, and crop litchi images to expand the dataset size; The processed litchi images are divided into training sets and test sets according to a preset ratio to establish a litchi image database.

9. A litchi image classification method based on the YOLO11 model according to claim 8, characterized in that, The YOLO11 model is trained using the Litchi image database, specifically including: Use the training set to train the model and monitor the accuracy and TOP-1 curve of the YOLO11 model in real time during the training process; Use the test set to evaluate the performance of the YOLO11 model, adjust the model parameters based on the evaluation results, and improve the model performance by optimizing the learning rate and batch size of the training process; After multiple iterations and tuning, a trained YOLO11 model is obtained.

10. A processing terminal, characterized in that: The invention comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the processing terminal executes a litchi image classification method based on a YOLO11 model as described in any one of claims 1 to 9.

Images