A method, device, equipment, medium and product for counting small tomatoes based on a YOLOv8 network

Abstract

The application discloses a small tomato counting method and device based on a YOLOv8 network, equipment, a medium and a product, relates to the technical field of target detection, and the method comprises the following steps: acquiring a video of small tomatoes with different degrees of maturity in a scale planting which is recorded in advance; inputting single-frame images in the video into a target detection model for detection to obtain the types of the small tomatoes and target frames; the target detection model is obtained by training an improved YOLOv8 model by using a training sample set, and the types of the small tomatoes include green fruits, red fruits and yellow fruits; different types of small tomatoes are counted according to the types of the small tomatoes and the target frames to obtain the quantities of the small tomatoes of various types. The application can improve the detection result of the small tomatoes, and further improve the counting efficiency and accuracy of the small tomatoes.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a method, apparatus, device, medium and product for counting small tomatoes based on the YOLOv8 network. Background Technology

[0002] Cherry tomatoes, a delicious and nutritious crop, have become an important variety in the global vegetable trade. Given their significant economic value, accurate fruit count is crucial for producers and operators. Precise fruit count prediction allows producers to better adjust their planting strategies, while operators can optimize operational decisions and increase profits. However, the growth characteristics of cherry tomatoes make fruit counting a complex task.

[0003] In recent years, with the rapid development of deep learning and computer vision technologies, object detection technology has made significant progress in the application of agriculture. In particular, deep learning-based object detection algorithms are widely used in fruit detection and counting. Object detection technology, through methods such as image classification and object localization, can accurately identify and locate target objects in complex backgrounds, thus providing a new solution for the detection and counting of agricultural fruits. While various improvements to object detection algorithms exist for specific application scenarios, the detection accuracy of cherry tomatoes remains low due to their unique growing environment and cultivation characteristics, resulting in low statistical efficiency and accuracy in cherry tomato counting. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, medium, and product for counting cherry tomatoes based on a YOLOv8 network, which can improve the detection results of cherry tomatoes and thus improve the counting efficiency and accuracy of cherry tomatoes.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a method for counting small tomatoes based on a YOLOv8 network, including:

[0007] Acquire pre-recorded videos of cherry tomatoes at different stages of ripeness during large-scale cultivation;

[0008] Single-frame images from the video are input into an object detection model for detection, resulting in the types of cherry tomatoes and their bounding boxes. The object detection model is an improved YOLOv8 model trained using a training sample set. This improved YOLOv8 model includes a backbone structure, a neck structure, and a detection head structure. The backbone structure includes a first ordinary convolutional module, multiple cross-convolutional modules, multiple Rep ghost bottleneck modules, and a feature pyramid pooling module. The neck structure includes a feature fusion module and a multiple attention mechanism module. The detection head structure includes four detection heads with different detection ranges. The types of cherry tomatoes include green, red, and yellow.

[0009] Count the different types of cherry tomatoes based on their varieties and the target box to obtain the quantity of each type of cherry tomato.

[0010] Secondly, this application provides a tomato counting device based on a YOLOv8 network, comprising:

[0011] The acquisition module is used to acquire pre-recorded videos of cherry tomatoes at different stages of maturity during large-scale cultivation.

[0012] The detection module is used to input single-frame images from the video into the object detection model for detection, thereby obtaining the types of cherry tomatoes and their bounding boxes. The object detection model is trained using a training sample set on an improved YOLOv8 model. The improved YOLOv8 model includes a backbone structure, a neck structure, and a detection head structure. The backbone structure includes a first ordinary convolutional layer, multiple cross-convolutional modules, multiple Rep ghost bottleneck modules, and a feature pyramid pooling module. The neck structure includes a feature fusion module and a multiple attention mechanism module. The detection head structure includes four detection heads with different detection ranges. The types of cherry tomatoes include green, red, and yellow.

[0013] The calculation module is used to count different types of cherry tomatoes based on the type of cherry tomato and the target box, and obtain the quantity of each type of cherry tomato.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described tomato counting method based on the YOLOv8 network.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described tomato counting method based on the YOLOv8 network.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described tomato counting method based on the YOLOv8 network.

[0017] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0018] This application provides a method, apparatus, device, medium, and product for counting small tomatoes based on a YOLOv8 network. By utilizing an improved YOLOv8 model to detect and identify single frames of pre-recorded small tomato videos, the method obtains the type of small tomato and its bounding box. The improved YOLOv8 model employs multiple cross-convolutional modules and a Rep ghost bottleneck module for feature extraction, making the network structure lightweight while ensuring rich features. A feature pyramid pooling module is used to further extract features, improving the detection capability of fine-grained features and providing richer feature representations for small tomato recognition. A multi-attention mechanism module is used to adjust the channel and spatial weights of the feature map, enabling efficient extraction of important information and effective feature fusion at different levels, improving the recognition accuracy of small targets and thus improving the overall target detection accuracy. Based on this, counting different types of small tomatoes can improve the counting efficiency and accuracy. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an application environment diagram of a tomato counting method based on a YOLOv8 network in one embodiment of this application;

[0021] Figure 2 A flowchart illustrating a method for counting small tomatoes based on a YOLOv8 network, provided as an embodiment of this application;

[0022] Figure 3 A schematic diagram of the structure of an improved YOLOv8 model provided in this application;

[0023] Figure 4 A schematic diagram illustrating a cross-convolutional module feature map processing method provided in this application;

[0024] Figure 5The diagrams provided in this application illustrate various improved network structures related to ghost modules, wherein (a) is a ghost module, (b) is a Rep ghost module, (c) is a ghost bottleneck module, and (d) is a Rep ghost bottleneck module.

[0025] Figure 6 The following are schematic diagrams of various variants of the feature pyramid pooling module provided in this application, wherein (a) is a traditional SPPF pyramid structure, (b) is a pyramid structure combining average and maximum, (c) is a single-branch pyramid combined with average pooling structure, (d) is a two-branch pyramid structure, and (e) is a two-branch symmetrical pyramid structure.

[0026] Figure 7 A schematic diagram illustrating the processing of feature maps by a multi-attention mechanism module provided in this application;

[0027] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] The tomato counting method based on the YOLOv8 network provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send pre-recorded videos of cherry tomatoes at different maturity levels from large-scale cultivation to server 104. After receiving the pre-recorded videos, server 104 inputs single-frame images from the videos into a target detection model for detection, obtaining the cherry tomato types and target boxes. Based on the cherry tomato types and target boxes, server 104 counts the different types of cherry tomatoes to obtain the quantity of each type. Server 104 can then feed back the obtained quantities of each type of cherry tomato to terminal 102. In addition, in some embodiments, the cherry tomato counting method based on the YOLOv8 network can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly process pre-recorded videos of cherry tomatoes at different maturity levels in large-scale planting, or the server 104 can obtain pre-recorded videos of cherry tomatoes at different maturity levels in large-scale planting from the data storage system for processing.

[0031] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, and tablets. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0032] In one exemplary embodiment, such as Figure 2 As shown, a method for counting small tomatoes based on a YOLOv8 network is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 203.

[0033] in:

[0034] Step 201: Obtain pre-recorded videos of cherry tomatoes at different stages of maturity during large-scale cultivation.

[0035] Step 202: Input a single frame image from the video into the object detection model for detection to obtain the types of cherry tomatoes and the target bounding boxes; wherein, the object detection model is obtained by training an improved YOLOv8 model using a training sample set, and the improved YOLOv8 model includes a backbone structure, a neck structure, and a detection head structure; the backbone structure includes a first ordinary convolutional module, multiple cross-convolutional modules, multiple Rep ghost bottleneck modules, and a feature pyramid pooling module, the neck structure includes a feature fusion module and a multiple attention mechanism module, and the detection head structure includes four detection heads with different detection ranges; the types of cherry tomatoes include green tomatoes, red tomatoes, and yellow tomatoes.

[0036] Step 203: Count the different types of cherry tomatoes according to the type of cherry tomato and the target box to obtain the quantity of each type of cherry tomato.

[0037] By implementing steps 201 to 203 above, this application can improve the counting efficiency and accuracy of cherry tomatoes.

[0038] In one exemplary embodiment, such as Figure 3 As shown, the traditional internal Bottleneck structure is replaced with Repghostbottleneck in the backbone structure to form C2f_Repghost (i.e., Rep ghost bottleneck module) module, and a feature pyramid pooling module is added. Therefore, the improved backbone structure includes a first ordinary convolution module, multiple cross-convolution modules, multiple Rep ghost bottleneck modules and a feature pyramid pooling module.

[0039] In the neck structure, the traditional internal Bottleneck structure GhostBottleneck is transformed into a C2f_Ghost module (i.e., a ghost bottleneck module), and a multi-attention mechanism module is added. Therefore, the improved neck structure includes a feature fusion module and a multi-attention mechanism module, and the feature fusion module contains multiple ghost bottleneck modules.

[0040] For the detection head structure, an additional detection head is added to the traditional detection head to detect minute features. Therefore, the improved detection head structure includes four detection heads with different detection ranges, namely the first detection head, the second detection head, the third detection head, and the fourth detection head, and the detection range of the first detection head, the second detection head, the third detection head, and the fourth detection head decreases in size sequentially.

[0041] Before step 202, the improved YOLOv8 model needs to be trained using training samples to obtain the target detection model. In this embodiment, a cherry tomato cultivation base in an agricultural expo park was selected as the experimental site. Cherry tomatoes at different growth stages were cultivated in the base for market supply, and the cherry tomatoes in the cultivation base were collected.

[0042] A SCOUT 2.0 robot was used, with a smartphone placed horizontally to photograph cherry tomatoes. Each row of tomatoes was 20 meters long, with two rows of plants, totaling 160 plants. During data collection, the robot moved at a constant speed, photographing each row of tomatoes from one direction. Single frames from the captured videos were standardized to 2160*3840 pixels, stored in JPG format, and labeled using the LabelImg tool. The labels were: green tomatoes, red tomatoes, and yellow tomatoes. A total of 661 cherry tomato data samples were obtained, which were divided into training, validation, and test sets in a 3:1:1 ratio.

[0043] In this embodiment, the mosaic technique built into the YOLOv8 network is used to enhance single-frame images to increase the model's robustness. Since the training set contains only 535 single-frame images, and these images may suffer from overexposure or blurring under natural lighting, this embodiment also employs six data augmentation techniques—rotation, random brightness adjustment, noise reduction, translation, mirroring, and cropping—to process the tomato data samples in the training set. Each augmentation uses three of these six techniques randomly combined. After data augmentation, the number of training samples is expanded to 5282. These expanded samples are then used as training samples to train the improved YOLOv8 model.

[0044] In object detection, the Intersection over Union (IOU) is used to calculate the overlap ratio between the predicted bounding box and the ground truth bounding box. However, the sensitivity of IOU to objects of different sizes varies significantly. For example, for a small 4x4 pixel object, a small positional deviation can lead to a significant decrease in IOU, while for a larger 45x45 pixel object, the change in IOU is smaller with the same positional deviation. This situation leads to insufficient learning of features for small objects by the object detection model, or even stalls the training process, preventing the model from being fully optimized. The sensitivity of IOU to different object sizes mainly stems from the unique characteristic that the bounding box position can only change discretely. Therefore, in this embodiment, the loss function used during training is:

[0045]

[0046] Where λ1 and λ2 are hyperparameters, f eIOU The loss is calculated for the predicted bounding box and the ground truth bounding box, where M is a constant and μ is a constant. a Let μ be the two-dimensional Gaussian distribution function of the true bounding box. b Let Gaussian distribution function be the two-dimensional distribution function of the predicted target bounding box. This represents the distribution distance between the predicted bounding box and the ground truth bounding box.

[0047]

[0048] Among them, I oU The cross-union ratio (CUP) of the predicted bounding box and the ground truth bounding box is represented by b and b'. gt Let w and w represent the center points of the predicted bounding box and the ground truth bounding box, respectively. gt Let h and h represent the widths of the predicted bounding box and the ground truth bounding box, respectively. gt Let α represent the heights of the predicted bounding box and the ground truth bounding box, respectively. 2 (·) represents Euclidean distance, d w and d h These represent the width and height of the minimum bounding box that covers the predicted target box and the actual target box, respectively.

[0049] In real-world planting scenarios, cherry tomato data samples are often labeled using rectangular bounding boxes. The cherry tomato itself, along with other background information, is distributed around the center and edges of the box, and the pixel weights decrease from the center towards the edges. Therefore, we abstract the predicted target box and use its inscribed circle to represent the different pixel weight distributions within it. Let the predicted target box R = (x... c ,y c ,w,y), where x c and y c ... x ,μ y ) represents the coordinates of the center of the ellipse, ρ x , ρ y Let μ be the semi-axis lengths of the ellipse along the x and y axes, respectively. x =x c μ y =y c , The corresponding equation of the ellipse is:

[0050]

[0051] For a p-dimensional random vector X = (X1, ..., X2) p ) T The probability density function can be written as:

[0052]

[0053] The probability density function is distributed as a p-variable normal distribution, denoted as X ~ N(μ, Σ), where Σ -1 Let |∑| denote the inverse matrix of Σ, |∑| denote the determinant of Σ, and T denote the transpose.

[0054] Based on the Mahalanobis distance, when (X-μ) T ∑ -1 When (X-μ)=1, the equation of the ellipse is a two-dimensional Gaussian distribution of contour lines. At this time, the predicted target box R=(x c ,y c (w,h) can be modeled as a two-dimensional Gaussian distribution. And the predicted target box A(x) a ,y a ,w a ,h a ) and the true target bounding box B(x) b ,y b ,w b ,h b The similarity between A and B can be converted into the distribution distance between two Gaussian distributions. For the two-dimensional Gaussian distributions μ of A and B, a =N(m1,Σ1) and μ b =N(m2,Σ2), and the two-dimensional Wasserstein Distance between them is defined as:

[0055]

[0056] Where m1 and m2 represent the mean vectors composed of the coordinates of the predicted target box A and the ground truth target box B, respectively, and μ a μ b Let C represent the two-dimensional Gaussian distribution function of A and B, and let C represent the center point. a Let cy represent the center x-axis coordinate of A. a Represents the center x-axis coordinate of A, ||·|| F Let T be the F-norm of the matrix. r This represents the rank of the matrix.

[0057] Finally, for After normalization, the final metric is obtained, and the final E is calculated. WDIOU The formula was used to compare the effects of different values ​​of M on the results through experiments, and the best results were achieved when M=1.0.

[0058] The performance of the target detection model of the small tomato was comprehensively evaluated, and the accuracy (P), recall (R), mean precision (mAP), floating point operations per second (FLOPs), and frames per second (FPS) were selected as the main evaluation criteria.

[0059]

[0060]

[0061]

[0062]

[0063] The calculation of accuracy (P) and recall (R) requires four values: true positives (TP), false positives (FP), and false negatives (FN). A positive sample (TP) is a small tomato data sample correctly identified by the object detection model. False positives (FP) and false negatives (FN) represent the number of actual tomato data samples that were incorrectly identified or missed by the object detection model. Accuracy (P) is the proportion of all predicted tomato targets correctly identified by the object detection model, and recall (R) is the proportion of all actual tomato targets correctly identified by the object detection model. For each category of tomato, a PR curve can be plotted, where AP represents the area under the PR curve; the closer the area is to 1, the better the performance. mAP is the average AP across all target categories and is the most commonly used evaluation metric in object detection, intuitively reflecting the model's performance. Q in the mAP formula represents the number of tomato categories in the validation set. FLOPs represent the computational cost of the model and are often used to measure the complexity of an algorithm or model. FPS evaluates the speed of object detection, i.e., the number of images that can be processed per second; the higher the FPS, the faster the object detection model.

[0064] In an exemplary embodiment, step 202 specifically includes steps 11-17:

[0065] Step 11: Input the single-frame image into the first ordinary convolution module for feature extraction to obtain the first feature map.

[0066] The size of a single frame image is 640*640*3. It is input into the first ordinary convolutional module for feature extraction, resulting in a first feature map with a size of 320*320*64.

[0067] Step 12: Input the first feature map into multiple cross-convolutional modules and multiple Rep ghost bottleneck modules to obtain feature maps of different sizes.

[0068] Specifically, feature maps of different sizes include second, third, fourth, and fifth feature maps, and the connection relationships between multiple convolutional modules and multiple Rep ghost bottleneck modules are shown, such as... Figure 3 As shown. Therefore, step 12 specifically includes steps 121-124:

[0069] Step 121: Input the first feature map into the first cross-convolution module to obtain the first deep fusion feature map, and input the first deep fusion feature map into the first Rep ghost bottleneck module to obtain the second feature map.

[0070] Step 122: Input the second feature map into the second cross-convolution module to obtain the second deep fusion feature map, and input the second deep fusion feature map into the second Rep ghost bottleneck module to obtain the third feature map.

[0071] Step 123: Input the third feature map into the third cross-convolution module to obtain the third deep fusion feature map, and input the third deep fusion feature map into the third Rep ghost bottleneck module to obtain the fourth feature map.

[0072] Step 124: Input the fourth feature map into the fourth cross-convolution module to obtain the fourth deep fusion feature map, and input the fourth deep fusion feature map into the fourth Rep ghost bottleneck module to obtain the fifth feature map.

[0073] In one exemplary embodiment, each cross-convolutional module consists of a space-to-depth and conv (SPDC) layer and a layer without convolutional stride.

[0074] The process of processing a feature map across any convolutional module will be illustrated as an example, such as... Figure 4 As shown, let X be the feature map with input size L×L×C1, where L represents the size of the first feature map and C1 represents the number of channels through the spatial to depth convolutional layer. Feature map X can be divided by all scales to obtain feature submaps f. x,y The scale value is preset by the user and is usually an even number. For example, when scale = 2, four feature submaps F are obtained. x,y The shape of each feature sub-map is Concatenate all feature sub-maps along the channel dimension to obtain a new feature map. The new feature map is then fed into a C2-free stride-less layer with a C2 filter (C2). <scale 2 C1) yields the deep fusion feature map.

[0075] For example, the input feature map X, with a size of 320*320*64, is processed by any one of the multiple cross-convolutional modules to obtain a deep fusion feature map with a size of 160*160*512.

[0076] In one exemplary embodiment, such as Figure 5 As shown in (b) and (d), the Rep Ghost Bottleneck Module in the backbone structure includes two Rep Ghost Modules. The feature map output by the SPDC Module is input to the Rep Ghost Bottleneck Module for processing.

[0077] Specifically, the input feature map is first processed by ordinary convolution, and then enhanced by two Rep ghost modules. Each Rep ghost module splits the feature map after ordinary convolution along the number of channels. One part of the split feature map is used for batch normalization (i.e., residual operation), and the other part is used for depthwise separable convolution (i.e., dilated convolution operation). Then, the outputs of the two parts are summed to implicitly fuse the features, and finally a new feature map is output. Ordinary convolution is added between the two Rep ghost modules to adjust the number of channels and ensure channel alignment.

[0078] For example, inputting a feature map with a size of 160*160*512 into the Rep ghost module results in a feature map with a size of 160*160*256. However, in this embodiment, two Rep ghost modules are used. Therefore, the feature map with a size of 160*160*512 is input into the Rep ghost bottleneck module and outputs a feature map with a size of 160*160*128.

[0079] Step 13: Input the fifth feature map into the feature pyramid pooling module to obtain the sixth feature map.

[0080] The Feature Pyramid Pooling-Fast (DASPPF) module includes a second ordinary convolutional module, a third ordinary convolutional module, a first average pooling module, a second average pooling module, a first max pooling module, a second max pooling module, and a third max pooling module. Specific step 13 includes steps 131-135:

[0081] Step 131: Input the fifth feature map into the second ordinary convolution module for feature extraction to obtain a depth feature map.

[0082] The second ordinary convolutional module has a kernel size of 1, a stride of 1, C4 input channels, and C4 / 2 output channels. After feature extraction in the second ordinary convolutional module, average pooling and max pooling techniques are introduced to minimize the neglect of local contextual information.

[0083] Step 132: Input the depth feature map into the first average pooling module and the second average pooling module in sequence to obtain the salient information feature map.

[0084] Step 133: Input the depth feature map sequentially into the first max pooling module, the second max pooling module, and the third max pooling module to obtain the global information feature map.

[0085] Step 134: The depth feature map, the salient information feature map, and the global information feature map are concatenated to obtain a multi-information fusion feature map.

[0086] Step 135: Input the multi-information fusion feature map into the third ordinary convolutional module for feature extraction to obtain the sixth feature map.

[0087] The third ordinary convolutional module has a kernel of 1, a stride of 1, 3C4 input channels, and C4 output channels.

[0088] like Figure 6 As shown, this embodiment adopts Figure 6 The DASPPF module shown in (d) processes the deep feature map. Specifically, it uses three parallel branches. The first branch does not perform any operation on the original feature information to preserve its integrity and enrich the feature information. The second branch performs a consistent-size max pooling operation to reduce the limitation on the input image size and strive to preserve significant feature information. Finally, the third branch fuses multi-level information through average pooling operations of different sizes, thereby enhancing the ability to extract global information. During use, the amount of padding is gradually adjusted according to the size of the feature map convolution kernel to expand the receptive field of the module itself. Finally, the third ordinary convolution module is used to restore the feature map to its original size to obtain the sixth feature map.

[0089] To find the optimal combination of average pooling and max pooling, a series of variations of the DASPPF module were tested, such as... Figure 6 From (a), (c), (d), and (e), find the most suitable effect to balance the advantages and disadvantages of average pooling and max pooling. Figure 6 (a) in the image represents the traditional SPPF feature pyramid structure. Figure 6 Based on the structure shown in (a), without adding any extra branches, we replace average pooling with max pooling to obtain... Figure 6 The structure shown in (b) is... Figure 6 The variant combination of DASPPF modules in the structure shown in (b) does not perform well. Therefore, an additional third branch is introduced to incorporate the average pooling operation, as follows: Figure 6 The structures shown in (c), (d), and (e) ultimately yield the best results when the third branch contains two average pooling operations for feature fusion. Figure 6 The structure shown in (d) is shown in the image. Figure 6The variant combination of the DASPPF module in the structure shown in (d) not only reduces the spatial dimension of the feature map and lowers the computational complexity of the model, but also smooths the feature map, reduces local noise, makes the extracted features more stable, improves the detection capability of fine-grained features, and provides richer feature representation for the recognition of small targets.

[0090] Step 14: Input the sixth feature map into the multi-attention mechanism module to obtain the seventh feature map.

[0091] The multi-attention mechanism module includes a positional attention module and a channel attention module. Specifically, it combines channel attention with a positional attention mechanism for non-local operations, such as... Figure 7 As shown, the input feature map is split into channels. After the split feature map is processed by channel attention and spatial attention, it is added to the other split feature map to obtain the output feature map. This can better integrate meaningful features in the channel and spatial dimensions and increase the effectiveness of network information.

[0092] Nonlocal operation network modules can realize the dependence of information at different locations on information at other locations, and the range of features obtained does not decrease with the increase of network depth. They can better focus on and enhance features at different locations at different levels of the network. For example, when given an input feature map F of size m×n, where m and n represent the length and width of the feature map respectively, a weight operation is applied to a point (i,j) on feature map F, where i and j represent the x and y coordinates of the point on feature map F respectively. This allows the input location element to perform calculation operations on its own location element. The general formula is expressed as:

[0093]

[0094] Where C(x) represents the standardized value after weight adjustment, f(x) i ,x j ) indicates that a weight adjustment operation is performed on this point, g(x) i The convolution operation is represented as 1×1×C5, where C5 represents the number of channels in the feature map of the input attention module. The weight adjustment operation includes three selection modes: Gaussian dot product mode, embedded Gaussian dot product mode, and connection mode. The similarity matrix f(x) is calculated through these three modes. i ,x j Finally, the response matrix F based on nonlocal features is obtained. i .

[0095] In an exemplary embodiment, the input sixth feature map is processed as follows:

[0096] The sixth feature map is processed by halving the number of channels to obtain the first feature map with halved channel number and the second feature map with halved channel number; the first feature map with halved channel number is processed sequentially through the channel attention module and the position attention module to obtain the salient information focusing feature map; the salient information focusing feature map and the second feature map with halved channel number are integrated to obtain the seventh feature map.

[0097] Halving the number of channels not only helps reduce computational redundancy and alleviate subsequent computational burden, but also effectively promotes selective feature focusing, making the subsequent attention mechanism more targeted and efficient. Positional and channel attention operations are applied to the feature map after the channel number is halved. The channel attention mechanism uses average pooling and max pooling operations to gather effective information, which is then shared with a multilayer perceptron (MLP) to effectively integrate the captured important features. This allows for adaptive weighting of features based on contextual information, expanding the receptive field, especially when local feature information is insufficient for small targets like tomatoes, effectively enhancing global information. Finally, the number of channels in the processed feature map is restored to its original size, preserving the network's ability to capture high-dimensional features while also reasonably optimizing computational efficiency and improving the model's generalization ability. The entire process is illustrated in the following formula.

[0098]

[0099] F (a,b) =γ(F1);

[0100]

[0101]

[0102]

[0103]

[0104] in, F represents the convolution operation. in This represents the sixth feature map of the input, γ represents the split operation that halves the number of channels, and F a F b This represents the first and second channel number halving feature maps after the channel number has been halved, F a1 F represents a The result after channel attention and summation with itself: CA represents applying channel attention, SA represents applying spatial attention, and σ represents the sigmoid operation. This indicates a max pooling operation. This indicates the average pooling operation. F represents the feature map addition operation. out This represents the seventh feature map.

[0105] The entire process effectively integrates channel attention and positional attention, enabling the network to not only efficiently extract important information but also to perform effective feature fusion at different levels, thereby improving the overall recognition accuracy.

[0106] Step 15: Use the seventh feature map as input to the fourth detection head.

[0107] Step 16: Input the second feature map, the third feature map, and the fourth feature map into the feature fusion module for upsampling, and use the upsampled second feature map, third feature map, and fourth feature map as the inputs of the first detection head, the second detection head, and the third detection head, respectively.

[0108] like Figure 3 As shown, in the feature fusion module, the sixth feature map is upsampled and then concatenated with the fourth feature map. After concatenation, the feature map is extracted through the bottleneck convolution module to obtain the processed fourth feature map.

[0109] The second and third feature maps are processed by concatenation and bottleneck convolution modules to obtain the processed second and third feature maps, respectively. The processed second feature map is used as the upsampled second feature map, and the upsampled second feature map is used as the input of the first detection head.

[0110] The processed second feature map is processed by a regular convolution module for feature extraction, concatenated with the processed third feature map, and then processed by a ghost bottleneck module to obtain an upsampled third feature map. The upsampled third feature map is used as the input to the second detection head.

[0111] The upsampled third feature map is processed by a regular convolution module for feature extraction and then concatenated with the processed fourth feature map. After processing by the ghost bottleneck module, the upsampled fourth feature map is obtained, which is then used as the input to the fourth detection head.

[0112] The feature fusion module includes two ghost bottleneck modules. Structurally, the ghost module is a basic component of the ghost bottleneck module, such as... Figure 5 As shown in (a) and (c), the Ghost Bottleneck Module integrates the Ghost Module into the conventional neck structure.

[0113] Specifically, the feature map is input into the ghost bottleneck module. First, it undergoes ordinary convolution processing. The convolution-processed feature map is then split along the number of channels. One part of the split feature map is left unprocessed, while the other part undergoes depthwise separable convolution (i.e., dilated convolution). The outputs of the two parts are then concatenated along the channel direction for feature fusion, resulting in a new feature map. This new feature map is then subjected to ordinary convolution to adjust the number of channels. The adjusted feature map is then split, with one part left unprocessed and the other part still undergoing depthwise separable convolution (i.e., dilated convolution). Finally, the two parts are concatenated along the channel dimension and summed for feature fusion, outputting the fused feature map.

[0114] Step 17: Obtain the type of cherry tomato and the target bounding box based on the detection results of the first detection head, the second detection head, the third detection head and the fourth detection head.

[0115] This application uses the YOLOv8 network as the baseline model and adjusts the backbone structure of the traditional YOLOv8 network by utilizing the C2f_Repghost module and SPDC layer. This enables the object detection model to extract more feature information while reducing computational cost and maintaining lightweight design, enhancing its ability to extract feature information from small objects. A new DASPPF module is proposed, using average pooling to reduce the impact of redundant information on effective features. Simultaneously, a multi-attention mechanism module is constructed, which splits the input information and fuses some parts using spatial and channel attention mechanisms to achieve feature fusion of feature maps at different scales. Furthermore, this application also proposes a new E... WDIOU The loss function formula uses a two-dimensional Gaussian distribution function to abstract the original IOU loss function, thus solving the problem of IOU's insensitivity to small targets. Finally, an additional small detector head is added to the detector head structure to enhance the extraction effect of small features, resulting in better recognition of small targets.

[0116] Based on the same inventive concept, this application also provides a YOLOv8-based tomato counting device for implementing the aforementioned YOLOv8-based tomato counting method. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more YOLOv8-based tomato counting device embodiments provided below can be found in the limitations of the YOLOv8-based tomato counting method described above, and will not be repeated here.

[0117] In one exemplary embodiment, a tomato counting device based on a YOLOv8 network is provided, comprising:

[0118] The acquisition module is used to acquire pre-recorded videos of cherry tomatoes at different stages of maturity during large-scale cultivation.

[0119] The detection module is used to input single-frame images from the video into the object detection model for detection, thereby obtaining the types of cherry tomatoes and their bounding boxes. The object detection model is trained using a training sample set on an improved YOLOv8 model. The improved YOLOv8 model includes a backbone structure, a neck structure, and a detection head structure. The backbone structure includes a first ordinary convolutional layer, multiple cross-convolutional modules, multiple Rep ghost bottleneck modules, and a feature pyramid pooling module. The neck structure includes a feature fusion module and a multiple attention mechanism module. The detection head structure includes four detection heads with different detection ranges. The types of cherry tomatoes include green, red, and yellow.

[0120] The calculation module is used to count different types of cherry tomatoes based on the type of cherry tomato and the target box, and obtain the quantity of each type of cherry tomato.

[0121] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores pre-recorded videos of cherry tomatoes at different ripeness levels during large-scale cultivation. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a cherry tomato counting method based on a YOLOv8 network.

[0122] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0123] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0124] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0125] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0128] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for counting cherry tomatoes based on a YOLOv8 network, characterized in that, The tomato counting method based on the YOLOv8 network includes: Acquire pre-recorded videos of cherry tomatoes at different stages of ripeness during large-scale cultivation; Single-frame images from the video are input into an object detection model for detection, resulting in the types of cherry tomatoes and their bounding boxes. The object detection model is an improved YOLOv8 model trained using a training sample set. This improved YOLOv8 model includes a backbone structure, a neck structure, and a detection head structure. The backbone structure includes a first ordinary convolutional module, multiple cross-convolutional modules, multiple Rep ghost bottleneck modules, and a feature pyramid pooling module. The neck structure includes a feature fusion module and a multiple attention mechanism module. The detection head structure includes four detection heads with different detection ranges. The types of cherry tomatoes include green, red, and yellow. Count the different types of cherry tomatoes based on their varieties and the target box to obtain the quantity of each type of cherry tomato.

2. The tomato counting method based on YOLOv8 network according to claim 1, characterized in that, The detection head includes a first detection head, a second detection head, a third detection head, and a fourth detection head, and the detection range of the first detection head, the second detection head, the third detection head, and the fourth detection head decreases sequentially. Single-frame images from the video are input into an object detection model for detection, resulting in the types of cherry tomatoes and their bounding boxes. Specifically, this includes: The single-frame image is input into the first ordinary convolutional module for feature extraction to obtain the first feature map; The first feature map is input into multiple cross-convolutional modules and multiple Rep ghost bottleneck modules to obtain feature maps of different sizes; wherein, the feature maps of different sizes include a second feature map, a third feature map, a fourth feature map and a fifth feature map; The fifth feature map is input into the feature pyramid pooling module to obtain the sixth feature map; The sixth feature map is input into the multi-attention mechanism module to obtain the seventh feature map; The seventh feature map is used as the input to the fourth detection head; The second feature map, the third feature map, and the fourth feature map are input into the feature fusion module for upsampling, and the upsampled second feature map, third feature map, and fourth feature map are used as the inputs of the first detection head, the second detection head, and the third detection head, respectively. The types and target boxes of cherry tomatoes are obtained based on the detection results of the first, second, third, and fourth detection heads.

3. The tomato counting method based on YOLOv8 network according to claim 2, characterized in that, The Rep ghost bottleneck module includes a first Rep ghost bottleneck module, a second Rep ghost bottleneck module, a third Rep ghost bottleneck module, and a fourth Rep ghost bottleneck module, and the cross-convolution module includes a first cross-convolution module, a second cross-convolution module, a third cross-convolution module, and a fourth cross-convolution module; The first feature map is input into multiple cross-convolutional modules and multiple Rep ghost bottleneck modules to obtain feature maps of different sizes, specifically including: The first feature map is input into the first transconvolution module to obtain the first deep fusion feature map, and the first deep fusion feature map is input into the first Rep ghost bottleneck module to obtain the second feature map; The second feature map is input into the second transconvolution module to obtain the second deep fusion feature map, and the second deep fusion feature map is input into the second Rep ghost bottleneck module to obtain the third feature map; The third feature map is input into the third cross-convolution module to obtain the third deep fusion feature map, and the third deep fusion feature map is input into the third Rep ghost bottleneck module to obtain the fourth feature map; The fourth feature map is input into the fourth cross-convolution module to obtain the fourth deep fusion feature map, and the fourth deep fusion feature map is input into the fourth Rep ghost bottleneck module to obtain the fifth feature map.

4. The tomato counting method based on YOLOv8 network according to claim 2, characterized in that, The feature pyramid pooling module includes a second ordinary convolution module, a third ordinary convolution module, a first average pooling module, a second average pooling module, a first max pooling module, a second max pooling module, and a third max pooling module; The fifth feature map is input into the feature pyramid pooling module to obtain the sixth feature map, specifically including: The fifth feature map is input into the second ordinary convolutional module for feature extraction to obtain a depth feature map; The depth feature map is sequentially input into the first average pooling module and the second average pooling module to obtain the salient information feature map; The depth feature map is sequentially input into the first max pooling module, the second max pooling module, and the third max pooling module to obtain the global information feature map; The deep feature map, the salient information feature map, and the global information feature map are concatenated to obtain a multi-information fusion feature map; The multi-information fusion feature map is input into the third ordinary convolutional module for feature extraction, resulting in the sixth feature map.

5. The tomato counting method based on YOLOv8 network according to claim 2, characterized in that, The multi-attention mechanism module includes a positional attention module and a channel attention module; The sixth feature map is input into the multi-attention mechanism module to obtain the seventh feature map, specifically including: The sixth feature map is processed by halving the number of channels to obtain a first feature map with halved channel number and a second feature map with halved channel number. The first channel-number-halved feature map is processed sequentially through the channel attention module and the position attention module to obtain a salient information focused feature map; The seventh feature map is obtained by integrating the salient information focused feature map and the second channel number halved feature map.

6. The tomato counting method based on YOLOv8 network according to claim 1, characterized in that, The loss function used in the training process of the improved YOLOv8 model is: Where λ1 and λ2 are hyperparameters, f eIOU The loss is calculated for the predicted bounding box and the ground truth bounding box, where M is a constant and μ is a constant. a Let μ be the two-dimensional Gaussian distribution function of the true bounding box. b Let Gaussian distribution function be the two-dimensional distribution function of the predicted target bounding box. This represents the distribution distance between the predicted bounding box and the ground truth bounding box.

7. A cherry tomato counting device based on a YOLOv8 network, characterized in that, The tomato counting device based on the YOLOv8 network includes: The acquisition module is used to acquire pre-recorded videos of cherry tomatoes at different stages of maturity during large-scale cultivation. The detection module is used to input single-frame images from the video into the object detection model for detection, thereby obtaining the types of cherry tomatoes and their bounding boxes. The object detection model is trained using a training sample set on an improved YOLOv8 model. The improved YOLOv8 model includes a backbone structure, a neck structure, and a detection head structure. The backbone structure includes a first ordinary convolutional layer, multiple cross-convolutional modules, multiple Rep ghost bottleneck modules, and a feature pyramid pooling module. The neck structure includes a feature fusion module and a multiple attention mechanism module. The detection head structure includes four detection heads with different detection ranges. The types of cherry tomatoes include green, red, and yellow. The calculation module is used to count different types of cherry tomatoes based on the type of cherry tomato and the target box, and obtain the quantity of each type of cherry tomato.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the tomato counting method based on the YOLOv8 network according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the tomato counting method based on the YOLOv8 network as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the tomato counting method based on the YOLOv8 network as described in any one of claims 1-6.

Images