Lightweight tomato maturity detection method based on improved YOLOv10

Abstract

The invention discloses an improved YOLOv10-based light-weight tomato maturity detection method, which comprises the following steps of: shooting to obtain tomato images with different maturity in a real scene, and marking tomato maturity category information and dividing a data set; the method comprises the following steps: constructing a detection model, on the basis of YOLOv10, replacing an original backbone network part of the YOLOv10 with a lightweight network structure MobileNetV4, replacing an original feature fusion module of the YOLOv10 with a fusion improved cross-scale feature fusion module CCFM, and introducing a mixed channel attention mechanism MLCA; and training a detection model by using the divided training set, and inputting the collected tomato image into the trained detection model to obtain an image with a tomato maturity category information marker box. According to the invention, efficient detection of multi-scale tomato targets can be realized in a complex agricultural scene, and especially for the practical problems of leaf shielding, fruit overlapping, different maturity of fruits in the same cluster and the like, the detection precision and reliability are significantly improved.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision target detection, relates to automatic detection of fruit and vegetable maturity, and particularly to a lightweight tomato maturity detection method based on improved YOLOv10. Background Art

[0002] Traditional methods for detecting the maturity of tomatoes rely on manual judgment or mechanical detection based on simple physical features such as color and size. They are time-consuming, have large errors, and have high labor costs, and cannot meet the needs of efficient and precise production in modern agriculture. Existing target detection technologies perform poorly in complex agricultural scenarios, and have problems such as low accuracy and poor robustness. With the development of deep learning algorithms, traditional target detection algorithms are shifting towards high efficiency, intelligence, and lightweight. Target detection algorithms based on deep learning include two-stage detection and single-stage detection. Two-stage detection algorithms often have characteristics such as high computational complexity and large number of parameters. They usually need to rely on high-performance hardware equipment, which limits their application in resource-constrained agricultural scenarios. Compared with two-stage algorithms, single-stage algorithms have fast detection speeds, but they also have relatively low positioning and recognition accuracy, insufficient detection capabilities for small targets, and poor real-time performance. In view of the fact that tomato fruits are affected by natural conditions such as leaf occlusion, fruit overlap, and different maturity in the same cluster in real scenarios, advanced target detection technology is needed to solve this key problem in order to adapt to these complex environmental factors. Therefore, modern agriculture urgently needs an intelligent and reliable technical means that can strike a balance between lightness, precision and efficiency to adapt to the deployment needs of low-computing power equipment and improve the detection capabilities of multi-scale targets, especially small targets, to achieve automated monitoring of tomato maturity and precise picking. Summary of the invention

[0003] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a lightweight tomato maturity detection method based on improved YOLOv10, so as to achieve efficient detection of multi-scale tomato targets in complex agricultural scenarios, especially for practical problems such as leaf occlusion, fruit overlap, and different maturity of fruits in the same cluster, and significantly improve the detection accuracy and reliability.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is:

[0005] A lightweight tomato maturity detection method based on improved YOLOv10 includes the following steps:

[0006] Step 1: Take images of tomatoes of different maturity in real scenes, and annotate the tomato maturity category information and divide the data set;

[0007] Step 2: Build a detection model. The detection model is one or more of the following improvements made to YOLOv10:

[0008] Improvement 1: Use the lightweight network structure MobileNetV4 to replace the original backbone network part of YOLOv10;

[0009] Improvement 2: Use the fusion-improved cross-scale feature fusion module CCFM to replace the original feature fusion module of YOLOv10;

[0010] Improvement 3: introduce the mixed channel attention mechanism MLCA;

[0011] Step 3: Use the training set divided in step 1 to train the detection model in step 2, and input the collected tomato image into the trained detection model to obtain an image with a marked box of tomato maturity category information.

[0012] In one embodiment, in step 1, the acquired tomato images are unified and normalized in size, and then a bounding box is added to the tomato fruit in each image, and the acquired images are divided into six categories according to variety categories and different maturity levels, namely, ripe large tomatoes, semi-ripe large tomatoes, unripe large tomatoes, ripe small tomatoes, semi-ripe small tomatoes and unripe small tomatoes, and then divided into training sets, validation sets and test sets, and finally data enhancement operations are performed on the training set images.

[0013] In one embodiment, the improvement 1, the replaced backbone network part includes:

[0014] ConvBN module 1 is a standard convolution module that converts the input image into the initial feature representation, and the downsampling operation reduces the input image size by half;

[0015] ConvBN module 2, halves the feature map size and maintains the number of channels;

[0016] ConvBN module three, halves the feature map size and expands the number of channels;

[0017] UIB search block 1 performs high-level feature extraction and increases the dimension of feature channels by changing the expansion factor of the inverted residual structure;

[0018] UIB search block 2 performs high-level feature extraction and adjusts the convolution kernel to capture information at different scales by expanding the number of channels;

[0019] ConvBN module 4 performs global feature fusion;

[0020] The SPPF module uses pooling operations of different sizes to process and fuse objects of different scales, allowing the network to learn features from different scales;

[0021] The PSAMLCA module combines spatial and channel information in global and local information to provide stronger expression capabilities.

[0022] In one embodiment, the ConvBN module 2 and the ConvBN module 3 are both composed of a 3×3 channel-by-channel convolution ConvBN and a 1×1 point-by-point convolution ConvBN. The three single-channel feature maps obtained from the channel-by-channel convolution are output through convolution calculation using four convolution kernels of size 1×1×3 to output four feature maps.

[0023] In one embodiment, the improvement 2 integrates the improved cross-scale feature fusion module CCFM, introduces the CSPStage feature fusion module to replace the original RepC3 module in the CCFM structure, and the CSPStage module is processed by ConvBNAct for channel reduction and Simplify Rep is used to build a re-parameterized mechanism network, wherein ConvBNAct is a convolution module composed of multiple convolutions, batch normalization and activation functions, and Simplify Rep is a simplified Rep structure, using 3×3 convolution during training and 1×1 convolution during inference.

[0024] In one embodiment, the CSPStage feature fusion module performs the following operations:

[0025] First, the feature information of different scales is merged, and then the merged feature map is divided into two layers, the upper layer only performs one convolution operation, and the lower layer uses the Simplify Rep module to perform feature fusion after the convolution operation; then the Concat operation is used to splice the feature information of the upper and lower layers that have been processed separately, fusing information from different levels; finally, a fused feature map is output through the RepConv operation.

[0026] In one embodiment, the improvement 2 is to fuse the improved cross-scale feature fusion module CCFM, introduce a GhostConv module to replace the original Conv module in the CCFM structure; the GhostConv module performs the following operations:

[0027] First, ordinary convolution is used to compress the number of channels of the input feature map. Then, the feature map after channel compression is processed in two parts. The first is to perform linear transformation on each channel to obtain the Ghost feature map. The second is to perform identity mapping on the feature map. Finally, the Ghost feature map and the feature map of the identity mapping are spliced ​​in the channel dimension to obtain the final feature map.

[0028] In one embodiment, the fused improved cross-scale feature fusion module CCFM comprises:

[0029] Conv module 1, upsampling module 1, fusion module 1, GhostConv module 1, upsampling module 2, fusion module 2, GhostConv module 2, fusion module 3, downsampling module and fusion module 4; each fusion module consists of a CSPStage feature fusion module and a Concat module;

[0030] Among them, Conv module 1 compresses the number of channels of the output feature map of the backbone network, the upsampling module 1 enlarges the size of the output feature map of Conv module 1, and the Fusion module 1 concatenates the output feature map of the upsampling module 1 with the output feature map of UIB search block 1 in the backbone network in the channel dimension, and then inputs the GhostConv module 1 for lightweight convolution operation; the upsampling module 2 enlarges the size of the output feature map of GhostConv module 1, and the Fusion module 2 concatenates the output feature map of the upsampling module 2 with the output feature map of ConvBN module 3 in the backbone network in the channel dimension, and then inputs the GhostConv module 2 for lightweight convolution operation; the Fusion module 3 concatenates the output feature map of GhostConv module 2 with the output feature map of GhostConv module 1 in the channel dimension, and then inputs the downsampling module to reduce the spatial and channel dimensions; the Fusion module 4 concatenates the output feature map of the downsampling module with the output feature map of Conv module 1 in the channel dimension.

[0031] In one embodiment, in each Fusion module, the Concat module concatenates feature maps from different sources along the channel dimension, and then divides them into upper and lower layers for processing respectively. The upper layer performs a convolution operation, and after the lower layer performs the convolution operation, the Simplify Rep module is used to perform feature fusion.

[0032] In one embodiment, the improvement 3 introduces a hybrid channel attention mechanism MLCA, which is added to the PSA module of the Backbone part to form a PSAMLCA module; the processing flow of the PSAMLCA module is as follows:

[0033] First, the feature map is input and local average pooling and global average pooling are performed in sequence. Local pooling focuses on the features of the local area, while global pooling captures the statistical information of the entire feature map.

[0034] Secondly, the features processed by local average pooling are rearranged, transformed by a 1D convolution, and rearranged again; the features processed by global average pooling are rearranged, transformed by a 1D convolution, and restored to the original spatial dimension through an unpooling operation; the 1D convolution is used to compress the feature channel while keeping the spatial dimension unchanged;

[0035] Then, the two features obtained in the previous step are added together and restored to the original spatial dimension through the unpooling operation again;

[0036] Finally, the features obtained in the previous step are combined with the original input image through a multiplication operation to obtain the final output.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention adopts the MobileNetV4 architecture and combines it with the universal inverted bottleneck (UIB) search block to significantly improve the computational efficiency and inference speed of the model, so that the model can run efficiently on resource-constrained mobile devices and meet the needs of real-time detection of tomato maturity. In addition, the introduced cross-scale feature fusion module CCFM can effectively fuse feature information of different scales, improving the detection ability of tomatoes of different sizes and maturity. In tomato maturity detection, the size and shape of tomatoes change significantly with maturity. The CCFM module enhances the performance of the model in multi-scale target detection by effectively integrating features of different scales, especially greatly improving the recognition ability of small-scale tomatoes.

[0039] This paper introduces the hybrid channel attention mechanism MLCA to optimize the network's ability to focus on key channel features. In the task of tomato maturity detection, its color and texture are key factors in judging maturity. The MLCA mechanism enhances the accuracy of feature extraction and improves the ability to accurately classify tomatoes at different maturity stages by emphasizing channel information related to maturity.

[0040] The optimized structure and lightweight design adopted by the present invention effectively reduce the consumption of computing resources and are particularly suitable for deployment on mobile devices. In the context of smart agriculture, smart devices can be used for on-site real-time detection to quickly assess the maturity of tomatoes to assist agronomic management. Through multi-scale fusion, attention mechanism and optimized network design, the accuracy and robustness of tomato maturity detection are significantly improved, and a high detection accuracy can be maintained, providing more reliable support for agricultural production.

[0041] It has been verified that, under the tomato image dataset in the same real complex scene, compared with the baseline model, the detection model of the present invention not only brings performance improvement in tomato fruit detection, but also significantly reduces the number of parameters and calculation amount of the model, effectively solving the problems of low accuracy and high calculation complexity of tomato fruit recognition and detection in complex scenes, and providing a solution for application deployment on restricted devices in agricultural scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The present invention is a flowchart of a lightweight tomato maturity detection method based on improved YOLOv10.

[0043] Figure 2 It is a structural schematic diagram of the detection model in the present invention.

[0044] Figure 3 Schematic diagram of the structure of the improved cross-scale feature fusion module CCFM in the present invention.

[0045] Figure 4 It is a schematic diagram of the structure of the Fusion module of the present invention.

[0046] Figure 5 Schematic diagram of the MLCA structure of the mixed channel attention mechanism added in the present invention.

[0047] Figure 6 These are the detection results of the detection model of the present invention on ordinary tomatoes of different maturity levels.

[0048] Figure 7 The following are the detection results of cherry tomatoes of different maturity using the detection model of the present invention. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0050] The invention discloses a lightweight tomato maturity detection method based on improved YOLOv10. By acquiring tomato images of different maturity in real scenes and designing an optimized detection model structure, the invention can realize efficient detection of multi-scale tomato targets in complex agricultural scenes, especially for practical problems such as leaf occlusion, fruit overlap and different maturity of fruits in the same cluster, and significantly improve detection accuracy and reliability. The invention adopts a lightweight single-stage target detection algorithm, optimizes model parameters and computational complexity in resource-constrained environments, so as to meet the deployment requirements of low-computing hardware devices. At the same time, by improving the backbone network, adding an adaptive attention mechanism and optimizing the multi-scale feature fusion module, the detection capability of small targets and the robustness in complex scenes are effectively enhanced. In addition, in order to improve the adaptability of the model under different environmental conditions, a training method based on multiple data enhancement strategies and multi-task learning is designed, so that the model can more accurately identify tomato fruits of different maturity. It is helpful to improve agricultural production efficiency, reduce labor costs, and provide reliable technical support for the automation and intelligence of modern agriculture.

[0051] Specifically, the present invention designs a lightweight tomato fruit maturity detection model based on an improved YOLOv10 network. First, a tomato image dataset is manually collected, then an improved network model is designed and trained on the image dataset, and finally the optimal model is used to verify tomato fruits in real scenes. Figure 1 The technical roadmap of the present invention is as follows:

[0052] Step 1: Obtain data images of tomato fruits of different maturity in real scenes through manual photography and other methods.

[0053] During the implementation of the present invention, firstly, image datasets of different types of tomato fruits with different degrees of maturity are collected, and these image datasets are used to train and verify the improved YOLOv10 model. Since the maturity of tomatoes is affected by many factors, such as illumination, occlusion, overlap between fruits, etc., it is necessary to collect images in a variety of different scenes to ensure the diversity and representativeness of the dataset. A greenhouse planting environment is selected for image collection to ensure the consistency of image collection, and tomato fruits of different degrees of maturity may be affected by different lighting conditions in the greenhouse, so it is possible to simulate the illumination change scenes that may occur in real applications. Tomato fruits under different backgrounds are selected for shooting, such as occlusion with other plants, tomato leaves or background objects, to improve the robustness of the model, and by simulating these complex scenes, it is ensured that the model can identify occluded or overlapping tomato fruits.

[0054] High-resolution smartphones were used to obtain high-quality image data, ensuring that the captured images were clear and detailed enough so that the model could effectively learn the characteristics of tomato fruits. Tomato fruits of each maturity category were photographed from multiple different angles to ensure that the model could recognize tomato fruits from different perspectives. The images were taken at different times such as morning, noon, and evening to increase the diversity of the dataset.

[0055] Step 2: All images collected in step 1 are preprocessed, and then the tomato images taken are labeled with information about maturity categories using image annotation tools such as LabelImg. In the present invention, the collected images are divided into 6 categories according to variety categories and different maturity levels, and divided into training sets, validation sets, and test sets in proportion. Finally, data enhancement operations are performed on the training set images.

[0056] Before data annotation of the collected images, the original images need to be preprocessed for subsequent image annotation, data enhancement and model training. First, check all images, filter out blurred images, and unify the size of all images. In this embodiment, the size is adjusted to 640×640×3 to ensure the uniformity of the images and adapt to the input of the model. This operation not only reduces the memory usage during training, but also improves the training efficiency. Then the images are normalized to speed up the training process of the model and avoid the influence of data numerical differences on the convergence of the network.

[0057] For the supervised learning target detection algorithm in deep learning, a large number of images with labeled information are required. Therefore, labeling is also a very important step in the process of building a dataset. By using the image annotation tool LabelImg, a bounding box is added to the tomato fruit in each image, and the corresponding category is marked. According to the variety category and different maturity, the collected images are divided into b_ripe, b_semi_ripe, b_unripe, l_ripe, l_semi_ripe and l_unripe. These 6 categories represent ripe large tomatoes, semi-ripe large tomatoes, unripe large tomatoes, ripe small tomatoes, semi-ripe small tomatoes and unripe small tomatoes, respectively.

[0058] After labeling, on the premise of ensuring that the data of each category is distributed as evenly as possible, the tomato dataset is randomly selected to ensure the representativeness of the training set, validation set, and test set, and to avoid the impact of category imbalance on training and validation. Finally, the dataset is divided into training set, validation set, and test set in a ratio of 8:1:1.

[0059] In order to improve the generalization ability of the model and avoid overfitting, the training set images are subjected to data enhancement operations. Data enhancement not only increases the sample size of the training set, but also simulates various changes that may occur in the real environment, thereby improving the robustness of the model. The data enhancement operations used in the present invention include horizontal flipping, vertical flipping, center cropping, brightness change, contrast change, saturation change, and Gaussian noise, and two of the enhancement techniques are randomly selected to transform the image. In addition, the improved YOLOv10 model retains the Mosaic data enhancement operation performed by the original model during the training phase, so the data enhancement operation used in this embodiment is sufficient to meet the comprehensive and diverse features learned by the model, so that it can better adapt to the complex environmental changes in real scenes.

[0060] Step 3: Build a detection model.

[0061] The detection model of the present invention is based on the improved YOLOv10 model, such as Figure 2 As shown, it includes one or more of the following improvements made to YOLOv10:

[0062] Improvement 1: Use the lightweight network structure MobileNetV4 to replace the original backbone network part of YOLOv10 and introduce deep separable convolution.

[0063] Improvement 2: Use the fused and improved cross-scale feature fusion module CCFM to replace the original feature fusion module of YOLOv10 and build an efficient pyramid network. Specifically, the CSPStage feature fusion module is introduced to replace the original RepC3 module in the CCFM structure, and the GhostConv module is introduced to replace the ordinary Conv module in the structure.

[0064] Improvement 3: introduce the mixed channel attention mechanism MLCA to enhance the feature extraction capability of the network.

[0065] The above improvements are introduced one by one below.

[0066] For improvement 1, the present invention adopts a lightweight network MobileNetV4 to replace the backbone network of the original YOLOv10 model. MobileNetV4 is a general and efficient architecture designed for mobile devices. It not only has the advantages of lightweight, but also improves the feature extraction efficiency through its modular design. Its design includes deep separable convolution, which greatly reduces the number of parameters and computational complexity, while ensuring high feature expression capabilities. The most important module in the model network, the Universal Inverted Bottleneck (UIB) search block, is a unified and flexible structure that integrates the Inverted Bottleneck (IB), Conv Next, the Feedforward Network (FFN) and a new Extra Deep Convolution (ExtraDW) variant. The two optional deep convolutions in the UIB module have four possible instantiations. The Inverted Bottleneck (IB) performs spatial mixing on the extended feature activation, providing a larger model capacity, but the cost increases. Conv Next allows cheaper spatial mixing with a larger kernel size by performing spatial mixing before expansion. Extra DW is a new variant that can achieve a cheap increase in network depth and receptive field, providing the combined advantages of Conv Next and IB. FFN is a stack of two 1×1 point-wise convolutions (PWs) with activation and normalization layers in between. At each network stage, UIB provides flexibility to make temporary trade-offs between spatial and channel mixing, expanding the receptive field and maximizing computational utilization as needed.

[0067] After the replacement, the backbone network of MobileNetV4 effectively extracts the multi-scale features of tomatoes from shallow to deep through a reasonable hierarchical design. Its efficient feature layer contains more shallow convolution kernels that are sensitive to local information and deep convolution kernels that are sensitive to global context, ensuring that the features of different maturity areas of tomatoes can be effectively captured. According to the characteristics of the tomato maturity detection task, the structure of MobileNetV4 is appropriately adjusted, such as reducing the network depth to adapt to smaller target sizes and adding shallow feature layers to enhance the detection capability of small targets. At the same time, the feature downsampling ratio is optimized to ensure that more detailed information is extracted. In addition, the SE (Squeeze-and-Excitation) attention mechanism built into MobileNetV4 further optimizes the representation capability of channel features by rebalancing the importance of channels. This can effectively improve the perception of targets blocked by leaves and improve the detection robustness of the model in complex agricultural scenarios.

[0068] Specifically, in the embodiment of the present invention, the replaced backbone network part includes: ConvBN module 1, ConvBN module 2, ConvBN module 3, UIB search block 1, UIB search block 2, ConvBN module 4, SPPF module and PSAMLCA module arranged in order of input. Among them:

[0069] ConvBN module 1 is a standard convolution module. In this embodiment, the number of input channels is 3, the number of output channels is 32, the convolution kernel size is 3*3, and the stride is 2. The function is to convert the input image into the initial feature representation, and the downsampling operation halves the size of the input feature map. Assuming that the input image size of the backbone network part is 640*640*3, the output image size is 320*320*32.

[0070] ConvBN module 2, in this embodiment, includes channel-by-channel convolution ConvBN with a convolution kernel size of 3*3 and a stride of 2, and point-by-point convolution ConvBN with a convolution kernel size of 1*1 and a stride of 1. Channel-by-channel convolution ConvBN halves the size of the feature map, and the output size is 160*160*32. Point-by-point convolution ConvBN is used for feature channel adjustment or fusion. The role here is to maintain the number of channels and the size of the feature map, 160*160*64. The role of ConvBN module 2 is to expand the initial features, maintain the details of the preliminary features, and enhance the network's ability to detect simple patterns. The output image size after this layer is 160*160*32.

[0071] ConvBN module three, in this embodiment, includes channel-by-channel convolution ConvBN with a convolution kernel size of 3*3 and a stride of 2, and point-by-point convolution ConvBN with a convolution kernel size of 1*1 and a stride of 1. Channel-by-channel convolution ConvBN is used for channel expansion to adapt to more complex features, and the output size is 80*80*64. Point-by-point convolution ConvBN is used for adjustment of feature channels, and the output size is 80*80*128. The function of ConvBN module three is to expand and downsample the feature channels, halve the size of the feature map, and expand the number of channels to enhance the characterization capability. ConvBN module two and ConvBN module three use a 3*3 convolution kernel to capture local information with a smaller receptive field. The image size output after passing through ConvBN module three in this embodiment is 80*80*64.

[0072] UIB search block 1, after ConvBN module 3, the number of input channels is 64. The function of UIB search block 1 is high-level feature extraction. There is a 5*5 convolution kernel in this layer to capture larger receptive field information. In addition, the feature channel is dimensionally upgraded by changing the expansion factor of the inverted residual structure. After passing through this layer, the output image size is 40*40*96.

[0073] UIB search block 2 further enhances features by increasing the number of channels and adjusting the convolution mode. It extracts advanced features and adjusts the convolution kernel to capture information of different scales by expanding the number of channels. After UIB search block 1, the input channel number is 96. UIB search block 2 further extracts advanced features and flexibly adjusts the convolution kernel to capture information of different scales by expanding the number of channels. The output image size after this layer is 20*20*128.

[0074] ConvBN module 4 is used to perform global feature fusion. After passing this layer, the output image size is 20*20*1280.

[0075] The SPPF module is used to process and fuse objects of different scales by using pooling operations of different sizes, so that the network can learn features from different scales and perform better when detecting objects of different sizes. The feature map size output after this layer is 20*20*1024.

[0076] The PSAMLCA module combines spatial and channel information in global and local information, enhances the capture of global information, and provides stronger expression capabilities. The output feature map size after this layer is 20*20*1024.

[0077] In this embodiment, by integrating MobileNetV4 into the backbone network of YOLOv10, the model achieves lower computational complexity and parameter quantity while maintaining high detection accuracy, providing an efficient feature foundation for subsequent module improvements.

[0078] Depthwise Separable Convolution is composed of depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, and pointwise convolution is used to extract channel features. Suppose a color picture with a size of 64×64 pixels and 3 channels is input. Then for standard convolution, after a convolution layer containing 4 filters, each filter contains 3 kernels, and the size of each kernel is 3×3, 4 feature maps of the same size as the input are finally output. The number of parameters of the obtained convolution layer is N=4×3×3×3=108. The formula is expressed as:

[0079] Parameter quantity: D K *D K *M*N

[0080] Calculation amount: D K *D K *M*N*D F *DF

[0081] Among them, the size of the convolution kernel is: D K *D K , the input channel is M, the output channel is N, and the output feature map size is D F *D F .

[0082] For depthwise separable convolution, it is mainly composed of channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution is also called depthwise convolution. The difference from standard convolution is that the convolution kernel of depthwise convolution is single-channel mode, and it is necessary to perform convolution on each input channel to obtain an output feature map with the same number of input feature map channels. Therefore, a 3-channel image generates 3 feature maps after operation, and the number of parameters of the convolution part is N=3×3×3=27. Since the number of feature maps output by depthwise convolution is too small, it may affect the validity of information, so it is necessary to perform point-by-point convolution on the feature map obtained by channel-by-channel convolution. Point-by-point convolution is essentially a dimensionality increase using a 1×1 convolution kernel. From the 3 single-channel feature maps obtained by channel-by-channel convolution, 4 feature maps are output through convolution calculation with 4 convolution kernels of size 1×1×3, and the number of convolution layer parameters obtained is N=1×1×3×4=12. Therefore, the number of parameters obtained by depthwise separable convolution is N=27+12=39.

[0083] The formula is:

[0084] Parameter quantity: D K *D K *M+M*N

[0085] Calculation amount: D K *D K *M*D F *D F +M*N*D F *D F

[0086] Among them, the size of the convolution kernel is: D K *D K , the input channel is M, the output channel is N, and the output feature map size is D F *D F .

[0087] According to the parameter and computational formulas of standard convolution and depth-separable convolution, the ratio of the two convolutions can be obtained:

[0088] Parameter ratio:

[0089] Calculation ratio:

[0090] From the formula of the ratio of parameter quantity to calculation quantity, we can know that N is generally large. It can be ignored. If the common 3×3 convolution kernel is used, the number of parameters and computation of the depthwise separable convolution will drop to about one-ninth of the original. Therefore, compared with the traditional convolutional neural network, the depthwise separable convolution has fewer parameters, effectively reducing the parameters required for the convolution layer, and the operation speed is faster than the traditional convolution, with less computation, easy to implement and deploy to different platforms, and can simplify the computing model to achieve high-precision computing on smaller devices.

[0091] For improvement 2, the present invention adopts a fused and improved cross-scale feature fusion module CCFM (Cross-scale feature fusion) structure to replace the feature fusion module of YOLOv10. This module inserts a Fusion module composed of convolutional layers on the fusion path, and the convolutional layers in the module are responsible for fusing features of adjacent scales. This embodiment can effectively merge features of different scales to improve the model's ability to recognize targets of different sizes, and through a dynamic feature fusion strategy, effectively integrate information from different feature layers, and can improve the detection capability of small-scale objects while maintaining a high computational efficiency. It is suitable for devices with limited computing resources. The structure of the fused and improved CCFM is as follows: Figure 3 shown.

[0092] In this embodiment, in order to solve the problem of low utilization rate of target features and insufficient fusion expression ability, the CSPStage feature fusion module is introduced to replace the original RepC3 module in the CCFM structure, and the CSP (Cross Stage Partial) structure is used to reduce redundant gradient transfer, while improving feature expression ability, efficiently integrating multi-scale features, improving feature utilization efficiency and reducing the model's demand for computing resources. The CSPStage module is a convolution module composed of multiple convolutions, batch normalization and activation functions, and a reparameterization mechanism network is built by Simplify Rep. Simplify Rep is a simplified Rep structure, using 3×3 convolutions during training and 1×1 convolutions during inference, and reparameterizing features through a simplified convolutional network.

[0093] In the CSPStage feature fusion module, multi-scale feature information is first merged, and then the merged feature map is divided into upper and lower layers for separate processing. The upper layer performs only one convolution operation, and the lower layer performs feature fusion through the Simplify Rep module after the convolution operation. The fluidity of multi-scale feature information is maintained through data transmission between each layer of staged feature fusion operations. Finally, the Concat operation is used to splice the feature information of the upper and lower layers that have been processed separately, fuse information from different levels, and perform a RepConv operation to output a fused feature map. Ultimately, without incurring additional computational burden, high-level semantics and underlying spatial features are refined, the ability to flow information is retained, and more efficient feature expression and feature fusion are achieved, while improving recognition speed.

[0094] In the fusion module of this embodiment, the feature flow path is optimized to ensure that the flow of shallow and deep features is more efficient. For the overlapping and complex backgrounds that frequently appear in the tomato detection scene, the CCFM module can improve the segmentation effect of overlapping targets by strengthening the description ability of low-level features on boundaries.

[0095] In addition, considering the limited computing resources in the algorithm deployment and the lightweight requirements, the GhostConv module is introduced to replace the original ordinary Conv module in the CCFM framework structure to reduce the model parameters and memory usage. The main idea is to mine the feature information required by the algorithm at a lower computational cost during the feature fusion process by performing group convolution and finally concatenating two sets of feature maps. The GhostConv module is a lightweight convolution module, which is usually used to reduce the amount of calculation and parameters while maintaining a high feature expression capability, and generate more features in an efficient way.

[0096] In the GhostConv module, ordinary convolution is first used to compress the number of channels of the input feature map to generate a smaller feature map, usually reducing the number of convolution kernels to half. Then the feature map after channel compression is processed in two parts. The first is to perform a linear transformation on each channel to obtain a Ghost feature map. This method is more efficient in calculation and can still retain important feature information. The second is to perform an identity mapping on the feature map. Finally, the Ghost feature map and the identity mapping feature map are spliced ​​in the channel dimension to obtain the final feature map.

[0097] In comparison, conventional convolution is composed of convolution, batch normalization and nonlinear activation function. The difference between GhostConv and traditional convolution is that the linear transformation process is a convolution without batch normalization and nonlinear activation. It uses fewer convolution operations to generate the main features, and then generates supplementary features through cheap operations, which greatly improves the calculation efficiency of the model. It improves the efficiency of convolution operations while ensuring the required features, thereby improving the lightweight degree of the algorithm while ensuring a certain recognition accuracy.

[0098] Finally, in the embodiment of the present invention, reference Figure 3 As shown, the fused and improved cross-scale feature fusion module CCFM includes: Conv module 1, upsampling module 1, fusion module 1, GhostConv module 1, upsampling module 2, fusion module 2, GhostConv module 2, fusion module 3, downsampling module and fusion module 4. Among them:

[0099] Conv module 1 is a Conv operation after the output of the backbone network. Its function is to compress the number of channels of the output feature map of the backbone network. At this time, the size of the feature map is 20*20.

[0100] The upsampling module 1 enlarges the size of the output feature map of the Conv module 1, and the size of the feature map after the operation is 40*40. The purpose is to ensure that the output feature map of the UIB search block 1 of the fourth layer of the backbone network can be spliced ​​in the channel dimension after the Conv1*1 operation to fuse feature information of different scales.

[0101] Fusion module 1 is a fusion module that performs a concatenation operation on the channel dimension after the output feature map of the upsampling module 1 and the output feature map of the UIB search block 1 in the backbone network undergo a Conv1*1 operation to fuse feature information of different scales, thereby improving the network's ability to detect targets of different scales.

[0102] GhostConv module 1 is a lightweight convolution module that aims to reduce the amount of calculation and parameters while retaining effective features. It performs lightweight convolution operations on the output feature map of Fusion module 1.

[0103] Upsampling module 2 enlarges the size of the output feature map of GhostConv module 1, and the size of the feature map after the operation is 80*80. The purpose is to ensure that the output feature map of ConvBN module 3 in the third layer of the backbone network can be spliced ​​in the channel dimension after Conv1*1 operation to fuse feature information of different scales.

[0104] Fusion module 2 concatenates the output feature map of upsampling module 2 and the output feature map of ConvBN module 3 in the backbone network in the channel dimension after Conv1*1 operation to fuse feature information of different scales, thereby improving the network's ability to detect targets of different scales.

[0105] GhostConv module 2 is a lightweight convolution module that aims to reduce the amount of calculation and parameters while retaining effective features. It performs lightweight convolution operations on the output feature map of Fusion module 2.

[0106] Fusion module three, after Conv1*1 operation, concatenates the output feature map of GhostConv module two and the output feature map of GhostConv module one in the channel dimension to fuse feature information of different scales, thereby improving the network's ability to detect targets of different scales.

[0107] The downsampling module reduces the spatial and channel dimensions of the output feature map of the Fusion module 3 and fuses it with other scale features after downsampling.

[0108] Fusion module 4 concatenates the output feature map of the downsampling module with the output feature map of Conv module 1 in the channel dimension to fuse feature information of different scales, thereby improving the network's ability to detect objects of different scales.

[0109] The outputs of the above-mentioned Fusion module 2, Fusion module 3, and Fusion module 4 correspond to a detection head respectively.

[0110] like Figure 4 As shown, each Fusion module of the present invention is composed of a CSPStage feature fusion module and a Concat module. In each Fusion module, the Concat module splices feature maps from different sources along the channel dimension, and then divides them into two layers for processing respectively. The upper layer performs a convolution operation of Conv2, and the lower layer performs a convolution operation of Conv3, and then uses the Simplify Rep module to perform feature fusion.

[0111] Figure 3 The three input arrows on the left are the outputs of the third, fourth, and last layers of the backbone network after Conv1*1 from top to bottom. The purpose of these three Conv operations is to compress the number of channels of feature maps of different resolutions and semantic depths to facilitate subsequent multi-scale feature fusion, such as Concat operations. Figure 3 The three output arrows on the right point to the three detection heads in the Head part, which are used to detect large, medium and small objects.

[0112] For improvement 3, the hybrid channel attention mechanism (MLCA) introduced in the present invention is a lightweight mechanism for improving the feature representation capability in deep neural networks. By introducing the channel attention mechanism, it is added to the PSA module of the backbone network to form a PSAMLCA module. Considering channel and spatial attention at the same time, the model can optimize the weighted allocation of multi-layer features, effectively improving the ability to focus on target areas, especially small targets and occluded targets, and enhancing the network's feature extraction capability. The working principle of the PSAMLCA module is as follows: Figure 5 shown.

[0113] First, the input feature map (C, W, H) is processed by local average pooling (LAP) and global average pooling (GAP) in sequence, where local pooling focuses on the features of the local area, while global pooling captures the statistical information of the entire feature map. Secondly, the features processed by local average pooling are rearranged and transformed by a 1D convolution (Conv1d). Here, 1D convolution is used to compress the feature channels while keeping the spatial dimension unchanged. After 1D convolution, the features are rearranged again (Reshape) to adapt to subsequent operations. The features processed by global average pooling are rearranged and transformed by a 1D convolution, and restored to the original spatial dimension through the unpooling operation. Then, the two features obtained in the previous step are added and restored to the original spatial dimension again through the unpooling operation. Finally, the features obtained in the previous step are combined with the original input image through a multiplication operation to obtain the final output. This process is equivalent to a feature selection, which strengthens the focus on useful features. MLCA can dynamically adjust feature weights so that the detection network can maintain efficient perception capabilities for backgrounds and target areas of different complexities, thereby improving detection accuracy.

[0114] Step 4: Input the training set images obtained after data enhancement in step 2 into the detection model constructed in step 3 for iterative training, and finally output the model with the best detection effect.

[0115] In this embodiment, a data set after data preprocessing and data enhancement is used for model training. Before training, it is necessary to configure the training parameters of the improved model, set the input image size to 640×640, the training rounds to 200 rounds, the optimizer to stochastic gradient descent (SGD), the batch size and other initial training information. During the training process, the model will perform forward propagation based on the input image and label, calculate the predicted value of each target, and then perform back propagation to update the weight of the model by calculating the difference between the loss function and the actual label. The optimizer updates the weight of the model according to the gradient to minimize the loss function, and hyperparameters such as learning rate and momentum will affect the convergence speed and stability of the optimization process. A dynamic learning rate decay strategy is introduced to ensure that the model converges quickly in the early stage and finely optimizes in the later stage to balance the detection speed and accuracy. Mixed Precision Training is used to improve the training speed and reduce the memory usage, while ensuring the model accuracy. The early stopping mechanism is used to prevent overfitting and improve the generalization ability of the model. At the end of each epoch, the model is evaluated on the validation set to calculate indicators such as accuracy, recall, and mAP (mean Average Precision). If the performance of the validation set is poor, it can be optimized by adjusting the learning rate, changing the model structure, or increasing the data set.

[0116] This embodiment can obtain a lot of information about the model. The first is the weight folder, which contains two weight files, one is the final model weight file at the end of training, and the other is the model weight file with the best performance on the validation set. The weight parameters of the model during the training process are saved in the file; the second is the args.yaml file, which mainly saves the parameters specified during training; the third is the confusion matrix (ConfusionMatrix), which is used to analyze and evaluate the performance of the model; the fourth is the F1_Curve file, which shows the change of F1 value under different classification thresholds; the fifth is the P_curve file, which shows the relationship between Precision and Confidence. Generally speaking, as the confidence increases, the detection accuracy also increases; the sixth The first is the R_curve file, which shows the relationship between Recall and Confidence. Generally speaking, as the confidence increases, the recall rate decreases. The seventh is the PR_curve file, which shows the relationship between the precision and recall rate of the model under different classification thresholds. The closer the PR curve is to the upper right corner, the better the performance of the model is, and the more it can correctly identify positive samples. The eighth is the results.csv file, which records the parameter information of the model during the training process. The ninth is the results file, which shows the curves composed of the training loss, validation loss, mAP50, mAP50-95, metrics / precision, and metrics / recall of the model during the training process. In addition, there are also detection effect diagrams during the training process. Based on this information, we can clearly understand the performance of the constructed model in various aspects during the training process, help further analyze the performance of the model and optimize it, and improve the performance of the tomato maturity detection model.

[0117] The evaluation indicators used in this embodiment are common evaluation indicators in current target detection tasks, including precision (Precision, P), recall (Recall, R), mean average precision (mean Average Precision, mAP), and frames per second (Frames Per Second, FPS). The calculation formula is as follows:

[0118]

[0119] Among them, TP (True Positive) indicates the number of correctly detected positive samples, that is, the number of areas containing targets correctly identified by the model; FP (False Positive) indicates the number of falsely detected negative samples, that is, the number of areas without targets mistakenly determined by the model as targets; FN (False Negative) indicates the number of missed positive samples, that is, the number of areas containing targets that the model failed to correctly detect; Precision refers to the proportion of samples correctly predicted as positive samples among the identified samples, and the higher the precision, the better the recognition effect; Recall refers to the proportion of samples correctly predicted as positive samples among the identified positive samples; mAP50 refers to the average accuracy of all categories when the IoU threshold is 0.5; In addition, FPS refers to the number of frames processed by the algorithm per second, that is, the number of images detected per second. The larger the value, the faster the detection speed. When measuring the algorithm, the ultimate goal of balancing the above indicators is to make it more in line with the requirements of lightweight algorithms while ensuring accurate detection.

[0120] Step 5: Input the tomato image to be detected into the best detection model obtained in step 4, and output an image with a marked box with tomato maturity category information.

[0121] This embodiment inputs the image to be detected into the improved YOLOv10 model for forward reasoning to obtain the target detection results, including target category, confidence level and other information, and displays this information intuitively in the image for easy inspection and analysis. Figure 6 , Figure 7 The figure shows the visualization results of the detection of different types of tomato fruits under the improved YOLOv10 model of this embodiment. It can be seen that the maturity results of different tomatoes are accurately identified.

Claims

1. A lightweight tomato maturity detection method based on improved YOLOv10, characterized in that: The steps include: Step 1: Take images of tomatoes of different maturity in real scenes, and annotate the tomato maturity category information and divide the data set; Step 2: Build a detection model. The detection model is one or more of the following improvements made to YOLOv10: Improvement 1: Use the lightweight network structure MobileNetV4 to replace the original backbone network part of YOLOv10; Improvement 2: Use the fusion-improved cross-scale feature fusion module CCFM to replace the original feature fusion module of YOLOv10; Improvement 3: introduce the mixed channel attention mechanism MLCA; Step 3: Use the training set divided in step 1 to train the detection model in step 2, and input the collected tomato image into the trained detection model to obtain an image with a marked box of tomato maturity category information.

2. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 1 is characterized in that: In the step 1, the acquired tomato images are unified and normalized in size, and then a bounding box is added to the tomato fruit in each image, and the acquired images are divided into six categories according to the variety category and different maturity, namely, ripe large tomatoes, semi-ripe large tomatoes, unripe large tomatoes, ripe small tomatoes, semi-ripe small tomatoes and unripe small tomatoes, and then divided into training sets, validation sets and test sets, and finally data enhancement operations are performed on the training set images.

3. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 1 is characterized in that: The improvement 1, the replaced backbone network part, includes the following parts arranged in sequence along the input order: ConvBN module 1 is a standard convolution module that converts the input image into the initial feature representation, and the downsampling operation reduces the input image size by half; ConvBN module 2, halves the feature map size and maintains the number of channels; ConvBN module three, halves the feature map size and expands the number of channels; UIB search block 1 performs high-level feature extraction and increases the dimension of feature channels by changing the expansion factor of the inverted residual structure; UIB search block 2 performs high-level feature extraction and adjusts the convolution kernel to capture information at different scales by expanding the number of channels; ConvBN module 4 performs global feature fusion; The SPPF module uses pooling operations of different sizes to process and fuse objects of different scales, allowing the network to learn features from different scales; The PSAMLCA module combines spatial and channel information in global and local information to provide stronger expression capabilities.

4. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 3 is characterized in that: The ConvBN module 2 and the ConvBN module 3 are both composed of a 3×3 channel-by-channel convolution ConvBN and a 1×1 point-by-point convolution ConvBN. The three single-channel feature maps obtained from the channel-by-channel convolution are output through convolution calculation using four convolution kernels of size 1×1×3 to output four feature maps.

5. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 3 is characterized in that: The improvement 2 integrates the improved cross-scale feature fusion module CCFM and introduces the CSPStage feature fusion module to replace the original RepC3 module in the CCFM structure. The CSPStage module is processed by ConvBNAct for channel reduction and Simplify Rep is used to build a re-parameterized mechanism network, wherein ConvBNAct is a convolution module composed of multiple convolutions, batch normalization and activation functions, and Simplify Rep is a simplified Rep structure, which uses 3×3 convolution during training and 1×1 convolution during inference.

6. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 5 is characterized in that: The CSPStage feature fusion module performs the following operations: First, the feature information of different scales is merged, and then the merged feature map is divided into two layers, the upper layer only performs one convolution operation, and the lower layer uses the Simplify Rep module to perform feature fusion after the convolution operation; then the Concat operation is used to splice the feature information of the upper and lower layers that have been processed separately, fusing information from different levels; finally, a fused feature map is output through the RepConv operation.

7. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 5 or 6, characterized in that: The improvement 2 is to integrate the improved cross-scale feature fusion module CCFM, introduce the GhostConv module to replace the original Conv module in the CCFM structure; the GhostConv module performs the following operations: First, ordinary convolution is used to compress the number of channels of the input feature map. Then, the feature map after channel compression is processed in two parts. The first is to perform linear transformation on each channel to obtain the Ghost feature map. The second is to perform identity mapping on the feature map. Finally, the Ghost feature map and the feature map of the identity mapping are spliced ​​in the channel dimension to obtain the final feature map.

8. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 5 or 6, characterized in that: The fused improved cross-scale feature fusion module CCFM includes: Conv module 1, upsampling module 1, fusion module 1, GhostConv module 1, upsampling module 2, fusion module 2, GhostConv module 2, fusion module 3, downsampling module and fusion module 4; each fusion module consists of a CSPStage feature fusion module and a Concat module; Among them, Conv module 1 compresses the number of channels of the output feature map of the backbone network, the upsampling module 1 enlarges the size of the output feature map of Conv module 1, and the Fusion module 1 concatenates the output feature map of the upsampling module 1 with the output feature map of UIB search block 1 in the backbone network in the channel dimension, and then inputs the GhostConv module 1 for lightweight convolution operation; the upsampling module 2 enlarges the size of the output feature map of GhostConv module 1, and the Fusion module 2 concatenates the output feature map of the upsampling module 2 with the output feature map of ConvBN module 3 in the backbone network in the channel dimension, and then inputs the GhostConv module 2 for lightweight convolution operation; the Fusion module 3 concatenates the output feature map of GhostConv module 2 with the output feature map of GhostConv module 1 in the channel dimension, and then inputs the downsampling module to reduce the spatial and channel dimensions; the Fusion module 4 concatenates the output feature map of the downsampling module with the output feature map of Conv module 1 in the channel dimension.

9. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 8 is characterized in that: In each Fusion module, the Concat module concatenates feature maps from different sources along the channel dimension, and then divides them into two layers for processing respectively. The upper layer performs a convolution operation, and after the lower layer performs the convolution operation, the SimplifyRep module is used for feature fusion.

10. The lightweight tomato maturity detection method based on improved YOLOv10 according to claim 1, characterized in that: Improvement 3 introduces the hybrid channel attention mechanism MLCA, which is added to the PSA module of the Backbone part to form a PSAMLCA module; the processing flow of the PSAMLCA module is as follows: First, the feature map is input and local average pooling and global average pooling are performed in sequence. Local pooling focuses on the features of the local area, while global pooling captures the statistical information of the entire feature map. Secondly, the features processed by local average pooling are rearranged, transformed by a 1D convolution, and rearranged again; the features processed by global average pooling are rearranged, transformed by a 1D convolution, and restored to the original spatial dimension through an unpooling operation; the 1D convolution is used to compress the feature channel while keeping the spatial dimension unchanged; Then, the two features obtained in the previous step are added together and restored to the original spatial dimension through the unpooling operation again; Finally, the features obtained in the previous step are combined with the original input image through a multiplication operation to obtain the final output.

Images