A fruit counting method and system based on an improved long-nosed raccoon optimization algorithm

Abstract

The application discloses a fruit counting imaging method and system based on an improved long-nosed raccoon optimization algorithm, and the method comprises the following steps: acquiring training images and constructing a training set; based on a YOLOv7-tiny model, adding a CBAM attention mechanism module to a first branch and a second branch in an ELAN-T module respectively, and constructing a target detection model; training the target detection model, taking a WIoU v3 as a loss function, combining a chaotic function and a long-nosed raccoon optimization algorithm to optimize hyperparameters of the target detection model, and obtaining a final target detection model; combining the final target detection model and a target tracking algorithm to process a video to be measured, and completing a tracking counting task. The system comprises an image acquisition module, a model construction module, a training optimization module and a tracking counting module. Through the use of the application, the yield of fruits can be counted in real time. The application can be widely applied to the field of target counting.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target counting, in particular to a fruit counting method and system based on an improved long-nosed raccoon optimization algorithm. BACKGROUND

[0002] In the field of agriculture, the existing technical solutions have obvious limitations in immature fruit identification and counting. When using a two-stage target detection algorithm for identification, the detection speed will slow down due to an additional step compared to a single-stage target detection algorithm. Target detection algorithms are highly specific and need to be improved to adapt to the current task when facing different identification targets. Currently, most inventions of immature fruits have large differences in background color. For green immature fruits in the growth period, the background color is similar to the environment, and there are situations such as leaf shading and fruit overlapping, which increase the difficulty of target identification. For yield measurement, most inventions identify and count static immature fruit pictures, and there are fewer inventions for dynamic fruit identification and counting. SUMMARY

[0003] Therefore, the present application proposes a fruit counting method based on an improved long-nosed raccoon optimization algorithm to achieve dynamic identification and counting to meet the actual needs of yield measurement. The method includes the following steps:

[0004] Obtain training images, perform data augmentation on the training images, and construct a training set;

[0005] Based on the YOLOv7-tiny model, add CBAM attention mechanism modules to the first branch and the second branch in the ELAN-T module before the P3 detection head to construct a target detection model;

[0006] Introduce a chaotic function into the long-nosed raccoon optimization algorithm;

[0007] Train the target detection model based on the training set, use WIoUv3 as the loss function, and combine the improved long-nosed raccoon optimization algorithm to optimize the hyperparameters of the target detection model to obtain the final target detection model;

[0008] Combine the final target detection model and the target tracking algorithm to process the video to be measured, and complete the tracking and counting tasks in real time.

[0009] In some embodiments, the process of obtaining training images, performing data augmentation on the training images, and constructing a training set specifically includes:

[0010] Obtain fruit images in different environments, including sunny, cloudy, and leaf-shading scenes;

[0011] The collected fruit images are supplemented by adding noise, image inversion, vertical mirroring and horizontal mirroring;

[0012] The supplemented image set is divided into a training set and a test set.

[0013] In some embodiments, the improved long-nosed raccoon optimization algorithm is combined to optimize the hyperparameters of the target detection model, which specifically includes:

[0014] Setting related parameters;

[0015] Based on the chaotic mapping function, a chaotic sequence is generated, and related parameters are initialized;

[0016] Individuals are used for global search and local search, and the search range is further expanded using the Levy flight strategy, and the position is updated using the greedy strategy;

[0017] After position updating, the optimal solution in the current iteration is compared with the historical optimal solution, if the optimal solution in the current iteration is greater than the historical optimal solution, the optimal solution in the current iteration is updated as the historical optimal solution; otherwise, the historical optimal solution remains unchanged;

[0018] It is judged whether the termination condition is reached, and the optimal hyperparameters are fixed according to the historical optimal solution to obtain the final target detection model.

[0019] In some embodiments, the final target detection model and the target tracking algorithm are combined to process the to-be-measured video, and the real-time tracking counting task is completed, which specifically includes:

[0020] Based on the SORT algorithm, re-identification counting is introduced, and the StrongSORT target tracking algorithm is constructed;

[0021] The final target detection model is combined with the StrongSORT target tracking algorithm constructed by introducing re-identification counting to track and count the citrus fruits of the video.

[0022] The application also provides a fruit counting system based on the improved long-nosed raccoon optimization algorithm, which comprises:

[0023] An image acquisition module is configured to acquire images and construct a training set;

[0024] A model construction module is configured to construct a target detection model based on YOLOv7 -tiny A basic framework is configured to introduce a CBAM attention mechanism module to construct a target detection model;

[0025] A training optimization module is configured to train the target detection model based on the training set, and the hyperparameters of the target detection model are optimized based on the improved long-nosed raccoon optimization algorithm. v3As a loss function, the target detection model is subjected to hyperparameter optimization in combination with a chaotic function and a long-nosed raccoon optimization algorithm to obtain a final target detection model.

[0026] A tracking counting module is configured to process the video to be measured in combination with the final target detection model and a target tracking algorithm to complete a tracking counting task.

[0027] Based on the above scheme, the application provides a fruit counting method and system based on an improved long-nosed raccoon optimization algorithm, taking YOLOv7-tiny as a benchmark model, adding a CBAM attention mechanism in the ELAN-T module before the P3 detection head of YOLOv7-tiny to improve the feature extraction capability of the model for small targets and citrus green fruits in complex backgrounds. By using the WIoU v3 loss function, the harmful gradient of low-quality instances generated by citrus green fruit overlap and leaf shielding is reduced, further improving the detection capability of citrus green fruits in complex backgrounds. The StrongSORT target tracking algorithm is used to combine the algorithm with the target detection model to form a citrus green fruit target detection and target tracking algorithm, which can identify and track the citrus green fruits in the video stream of the citrus orchard, providing technical support for the yield measurement task of the citrus orchard to meet the actual needs of the yield measurement task. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a step flowchart of the fruit counting method based on the improved long-nosed raccoon optimization algorithm of the application;

[0029] Figure 2 is a structural schematic diagram of the improved ELAN-TC module;

[0030] Figure 3 is a structural block diagram of the fruit counting system based on the improved long-nosed raccoon optimization algorithm of the application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0032] It should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings. The embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0033] It should be understood that the "system", "apparatus", "unit" and / or "module" used in the present application is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0034] As shown in the present application and claims, unless the context clearly indicates otherwise, the words "one", "a", "an", and / or "the" do not refer to the singular, but can also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements explicitly identified, and these steps and elements do not constitute an exclusive list, and the method or device can also include other steps or elements. The element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, product or device comprising the element.

[0035] In the description of embodiments of the present application, "a plurality of" means two or more than two. The following terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0036] In addition, flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or subsequent operations are not necessarily performed in sequence. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can be added to these processes, or one or more steps of operation can be removed from these processes.

[0037] Reference Figure 1 The flowchart of an optional example of the fruit counting method based on the improved long-nosed raccoon optimization algorithm proposed in the present application can be applied to a computer device. The fruit counting method proposed in the present embodiment can include but is not limited to the following steps:

[0038] Step S1, acquiring an image and constructing a training set;

[0039] Step S2, based on the YOLOv7-tiny basic framework, introducing the CBAM attention mechanism module to construct a target detection model;

[0040] Step S3, training the target detection model based on the training set, taking WIoU v3 as the loss function, combining the chaotic function and the long-nosed raccoon optimization algorithm to optimize the hyperparameters of the target detection model, and obtaining the final target detection model;

[0041] Step S4, combining the final target detection model and the target tracking algorithm to process the video to be tested, and completing the tracking counting task.

[0042] In some feasible embodiments, the step S1 specifically comprises:

[0043] A handheld image acquisition device is used to acquire citrus green fruit pictures in sunny and cloudy environments, tree leaf blocking, clear and blurred scenes, etc.

[0044] The collected citrus green fruit images are added with pepper salt noise, image inversion, vertical mirror image, horizontal mirror image, etc. Data enhancement method is used to expand the collected citrus green fruit data, and finally the original image and the data enhanced image are used to construct the citrus green fruit data set.

[0045] The Labelimg tool is used to mark the citrus green fruit data set, and the marked data is divided into training set and verification set in the ratio of 8:2.

[0046] In some feasible embodiments, the step S2 specifically comprises:

[0047] YOLO-CW, a citrus green fruit target detection model, is proposed based on YOLOv7-tiny as the benchmark model.

[0048] The first branch and the second branch in the ELAN-T module in front of the P3 detection head are added with CBAM attention mechanism modules respectively, designed as ELAN-TC module, to improve the feature extraction capability of citrus green fruit in small target and complex background, and reduce the calculation amount and parameter amount of the model.

[0049] In order to further extract features in the deep layer, in the first branch, after channel and spatial convolution, a CBAM module is added,

[0050] Because the shallow feature information retained by one channel convolution is less, it is difficult to extract the features of small targets such as citrus green fruit and targets in complex background, therefore, in the second branch, after channel convolution, another CBAM module is added.

[0051] Because the CBAM module is added, the calculation amount and parameter amount of the ELAN-TC module are reduced compared with the ELAN-T module, and the structure diagram thereof is referred to Figure 2 .

[0052] The CBAM attention mechanism module includes a channel attention module (ChannelAttention Module, CAM) and a spatial attention module (SpartialAttention Module, SAM), and the two modules are independent of each other. First is the channel attention module.

[0053] M c (F) = σ(MLP(AvgPool(F)) + MLP(MaxPool(F))) # (1)

[0054] where F is the input feature map of the channel attention module, MLP is a two-layer neural network, M c is the final output of the channel attention module. The feature map is first passed through the channel attention module, and global maximum pooling and global average pooling are performed on the width and height respectively, and then the outputs of the MLP are added element-wise and activated by sigmoid to output M c .

[0055] M c and the original feature map are multiplied element-wise to serve as the input F' of the spatial attention mechanism.

[0056] M s (F') = σ(f 7×7 ([MaxPool(F'), AvgPool(F')])) # (2)

[0057] where F' is the input feature map of the spatial attention module, f 7×7 is a 7x7 convolution kernel, and M s is the final output of the spatial attention module. F' is subjected to global maximum pooling and global average pooling in the channel, then subjected to channel concatenation operation, and then reduced in dimension by 7x7 convolution operation and sigmoid activation operation to output M s , which is finally multiplied element-wise with F to complete the entire process. Since the CBAM module extracts the features of the target in the process of channel convolution and spatial convolution, it discards useless feature information, thereby reducing the computational load and parameter quantity brought by useless information.

[0058] In some embodiments, the loss function in step S3 is represented by the following formula:

[0059]

[0060] where x and y represent the positions of the center points of the prediction box, x gt and y gt represent the positions of the center points of the calibration box, W g and H g represent the width and height of the calibration box, is the overlap degree of the prediction box and the real box in the target detection task, is the monotonic focusing coefficient.

[0061] During the labeling of citrus green sample, the overlapping of fruits and the shielding of leaves will make the labeled citrus green incomplete. Although the CIoU used in YOLOv7-tiny considers the overlapping area, center point distance and aspect ratio of the boundary box regression, the difference in aspect ratio is not the real difference between the width and height and the confidence, which leads to the model to predict the overlapping or shielded fruits and background as citrus green during the training process, and thus aggravates the low-quality examples and reduces the generalization ability of the model. Therefore, the loss function CIoU is replaced by WIoU (Wise IoU) to solve the problem that the low-quality examples affect the training effect during the training process.

[0062] Therefore, in the present application, due to the overlapping of fruits and the shielding of leaves of citrus green, the labeled box of these samples contains less effective part of citrus green during the data labeling process, and the prediction box will label these samples and background during the training process. This makes the low-quality examples constantly affect the training process, and thus affects the generalization ability and detection ability of the whole model. In the present application, WIoU v3 is used to solve this problem. For the citrus green with overlapping fruits and leaf shielding, the gradient gain of low-quality examples is constantly adjusted during the training process, and thus the harmful gradient generated by low-quality examples is effectively prevented, so that the model focuses on ordinary examples and high-quality examples.

[0063] In some possible embodiments, the hyperparameter optimization process in the step S3 specifically comprises:

[0064] The improved long-nosed raccoon intelligent optimization algorithm (TCOA) is used to improve the YOLO-CW hyperparameters.

[0065] Inspired by the hunting behavior of long-nosed raccoons, the algorithm mainly includes the hunting and attacking strategy of raccoons to iguanas and the strategy of escaping from predators. First, half of the raccoon individuals climb a tree to hunt iguanas, and the other half of the raccoon individuals gather under the tree to wait for the iguanas to fall. When the iguanas fall, the raccoons kill them.

[0066] For the raccoons on the tree.

[0067]

[0068] where I is a random integer of [1, 2], N is the number of raccoon population, x i (j) represents the current i-th raccoon individual, x best (j) represents the optimal position of the current raccoon population.

[0069] For the iguanas and the raccoons on the ground.

[0070] Iguana ground (j) = lbj +randowm·(ub j -lb j ) #(8)

[0071]

[0072] where ub j , lb j are the upper and lower bounds of the current hyperparameters, Iguana ground (j) is the current position of Iguana, which represents the global optimal solution. fitness(Iguana ground ) and fitness(x i ) represent the fitness values of Iguana and the ith raccoon individual, x i (j) is the jth hyperparameter of the ith raccoon individual. Since the initialization process of the long-nosed raccoon optimization algorithm is random, it is easy to lose population diversity. Using chaotic mapping function to generate chaotic sequence for initialization can make the population distribution more uniform and improve search efficiency. The present application uses TENT chaotic function to redefine:

[0073] X i :x i,j = ub j +(ub j -lb j )·z i #(10)

[0074] where the TENT function expression is as follows, α = 0.5.

[0075]

[0076] where X i represents the hyperparameter group of the ith individual, x i,j represents the jth hyperparameter in the ith individual, lb j represents the lower bound of the jth hyperparameter in the ith individual, ub j represents the upper bound of the jth hyperparameter in the ith individual, and z i represents the ith mapping vector in the TENT sequence.

[0077] During the hunting process of long-nosed raccoon, it is easy to fall into local optimal solution. Using Levy flight strategy to update the position, let part of the long-nosed raccoon individuals to the wider search space.

[0078]

[0079] where S is the optimal solution of long-nosed raccoon, θ and ω obey normal distribution, σ u and σ vis the variance of the normal distribution of θ and ω, and the parameter β is a random number in [0,2].

[0080] To escape a predator, coatis will flee their original location and seek a safe location.

[0081]

[0082] Where t represents the current number of iterations, using the upper and lower limits of the current hyperparameters ub j , lb j Divide by t to get and The position of the current coati individual after escaping from the predator is calculated by formula (14). Each coati in the population uses the greedy strategy to update its position after moving.

[0083] Set relevant parameters, such as population size, number of iterations, upper and lower limits; initialize the initial position of each individual, and save the initial historical optimal solution X g,best ;

[0084] Each individual is searched globally using formula (7) and formula (9), and locally using formula (13) and formula (14). Then, formula (12) is used to further expand the search range to prevent the individual from entering the local optimum. Finally, the position is updated using the greedy strategy comparison. The value with the larger fitness value is selected for the final position update.

[0085] After the position is updated, the optimal solution X in the current iteration is selected best and the historical optimal solution X g,best For comparison, if X best >X g,best , then update the historical optimal solution, otherwise it remains unchanged.

[0086] After the termination condition is reached, the optimized optimal hyperparameter X g,best Train the model to obtain the final model.

[0087] In some embodiments, step S4 specifically includes:

[0088] Build the StrongSORT target tracking algorithm;

[0089] The SORT target tracking algorithm combines a target detector and a tracker. The target detector is a trained target detection model, and the tracker is a combination of Kalman filtering and the Hungarian algorithm. To achieve target tracking, Kalman filtering predicts the position and state of the next frame of the target movement, and the Hungarian algorithm matches and associates the predicted position and state with the actual detection result. The StrongSORT algorithm improves the SORT algorithm by introducing a re-identification technology. While focusing on target movement, the model extracts the appearance features of the target and compares them with the features stored in the previous frame to determine whether it is the same object, thereby improving the accuracy of tracking.

[0090] The YOLO-CW model is combined with the StrongSORT algorithm to track and count the citrus fruits in the video.

[0091] Based on the above scheme, to test the counting effect of the tracking and counting algorithm, the number of fruits, leaf shading, and complex background of citrus fruits in the citrus orchard were manually counted and detected, and the number of errors between manual counting and detection was calculated. The results are shown in Table 1. As can be seen from the table, in the case of obvious fruits, less shading, severe shading, and complex background, the error between manual counting and detection is within 3, which indicates that the counting ability of the algorithm is not much different from manual counting, and meets the requirements of counting citrus fruits in the citrus orchard.

[0092] Table 1: Relevant detection and counting data of citrus fruits by manual and algorithm

[0093] Actual number / individuals Model count / individuals Error number / individuals 9 8 1 16 15 1 16 18 2 35 34 1

[0094] As shown in Figure 3 , a fruit counting imaging system based on an improved long-nosed raccoon optimization algorithm includes:

[0095] An image acquisition module is used to acquire images and build a training set.

[0096] A model construction module is based on a YOLOv7-tiny basic framework and introduces a CBAM attention mechanism module to construct a target detection model.

[0097] A training optimization module trains the target detection model based on the training set, uses WIoU v3 as the loss function, combines a chaotic function and a long-nosed raccoon optimization algorithm to optimize the hyperparameters of the target detection model, and obtains a final target detection model.

[0098] A tracking and counting module is used to combine the final target detection model and a target tracking algorithm to process the video to be tested and complete the tracking and counting task.

[0099] The contents in the method embodiments are applicable to the system embodiments, the system embodiments specifically implement the functions same as the method embodiments, and achieve the beneficial effects same as the method embodiments.

[0100] A fruit counting imaging device based on an improved long-nosed raccoon optimization algorithm

[0101] At least one processor;

[0102] At least one memory for storing at least one program;

[0103] When the at least one program is executed by the at least one processor, the at least one processor implements the fruit counting imaging method based on the improved long-nosed raccoon optimization algorithm.

[0104] The contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically implement the functions same as the method embodiments, and achieve the beneficial effects same as the method embodiments.

[0105] A storage medium, wherein the storage medium stores processor-executable instructions, and the processor-executable instructions, when executed by a processor, are used to implement the fruit counting imaging method based on the improved long-nosed raccoon optimization algorithm.

[0106] The contents in the method embodiments are applicable to the storage medium embodiments, the storage medium embodiments specifically implement the functions same as the method embodiments, and achieve the beneficial effects same as the method embodiments.

[0107] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A fruit counting method based on an improved coati optimization algorithm, characterized in that: The following steps are involved: Get images and build a training set; Based on the YOLOv7-tiny basic framework, the CBAM attention mechanism module is introduced to build a target detection model; The target detection model is trained based on the training set, WIoU v3 is used as the loss function, and the target detection model is hyperparameter optimized by combining the chaotic function and the raccoon optimization algorithm to obtain the final target detection model; The final target detection model and target tracking algorithm are combined to process the video to be tested to complete the tracking and counting task; The formula of WIoU v3 loss function is as follows: Among them, x and y represent the position of the center point of the prediction box, x gt and y gt Represents the position of the center point of the calibration frame, W g and H g Represents the width and height of the calibration box, is the degree of overlap between the predicted box and the real box in the target detection task, is the monotonic focusing coefficient; The step of optimizing the hyperparameters of the target detection model by combining the chaotic function and the coati optimization algorithm specifically includes: Set relevant parameters; Generating a chaotic sequence based on a chaotic mapping function and initializing the relevant parameters; Use individuals to perform global and local searches and update positions; Select the optimal solution in the current iteration and compare it with the historical optimal solution, and determine the historical optimal solution based on the comparison results; When the termination condition is determined to be met, the optimal hyperparameters are fixed according to the historical optimal solution to obtain the final target detection model; The search also includes: Expand your search using the following formula: Where S is the optimal solution of the coati, θ and ω obey the normal distribution, σ u and σ v They represent the variance of the normal distribution of θ and ω respectively, and the parameter β is a random number in [0,2].

2. The fruit counting method based on the improved coati optimization algorithm according to claim 1, characterized in that: The step of building a target detection model based on the YOLOv7-tiny basic framework, introducing the CBAM attention mechanism module, specifically includes: Based on the YOLOv7-tiny framework, we added CBAM attention mechanism modules to the first and second branches of the ELAN-T module in front of the P3 detection head to build an object detection model. The CBAM attention mechanism module includes a channel attention module and a spatial attention module.

3. The fruit counting method based on the improved coati optimization algorithm according to claim 2, characterized in that: The initialization formula is as follows: X i :x i,j =ub j +(ub j -lb j )·z i Among them, X i represents the hyperparameter group of the i-th individual, x i,j represents the jth hyperparameter in the i-th individual, lb j represents the lower limit of the jth hyperparameter in the i-th individual, ub j represents the upper limit of the jth hyperparameter in the i-th individual, z i represents the i-th mapping vector in the TENT sequence, and α is a preset parameter.

4. The fruit counting method based on the improved coati optimization algorithm according to claim 3, characterized in that: in: The global search formula is as follows: Iguana ground (j)=lb j +random·(ub j -lb j ) Among them, Iguana ground (j) is the current position of the iguana, fitness(Iguana ground ) and fitness(x i ) represents the fitness value of the iguana and the i-th individual, x i (j) is the jth hyperparameter of the i-th individual; The formula for local search is as follows: Where t represents the current iteration number.

5. A fruit counting system based on an improved coati optimization algorithm, characterized in that: The method for executing the fruit counting method based on the improved coati optimization algorithm as claimed in claim 1 comprises: Image acquisition module, used to acquire images and build training sets; The model building module is based on the YOLOv7-tiny basic framework and introduces the CBAM attention mechanism module to build an object detection model; A training optimization module trains the target detection model based on the training set, uses WIoU v3 as the loss function, and combines the chaotic function and the raccoon optimization algorithm to optimize the hyperparameters of the target detection model to obtain the final target detection model; The tracking and counting module is used to process the video to be tested in combination with the final target detection model and the target tracking algorithm to complete the tracking and counting task.

6. A fruit counting device based on an improved coati optimization algorithm, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the fruit counting method based on the improved coati optimization algorithm as described in any one of claims 1 to 4.

Images