Fruit counting method and system based on improved raccoon optimization algorithm

Abstract

The invention discloses a fruit counting imaging method and system based on an improved raccoon longnose optimization algorithm, and the method comprises the steps: obtaining a training image, and constructing a training set; on the basis of a YOLOv7-tiny model, CBAM attention mechanism modules are added to a first branch and a second branch in an ELAN-T module respectively, and a target detection model is constructed; training the target detection model, and performing hyper-parameter optimization on the target detection model by taking WIoU v3 as a loss function and combining a chaos function and a raccoon optimization algorithm to obtain a final target detection model; and processing a to-be-detected video in combination with the final target detection model and a target tracking algorithm to complete a tracking counting task. The system comprises an image acquisition module, a model construction module, a training optimization module and a tracking counting module. By using the device, the yield of fruits can be counted in real time. The method can be widely applied to the field of target counting.

Description

Technical Field

[0001] The invention relates to the field of target counting, and in particular to a fruit counting method and system based on an improved coati optimization algorithm. Background Art

[0002] In the agricultural field, existing technical solutions have obvious limitations in the recognition and counting of immature fruits. When using a two-stage target detection algorithm for recognition, the detection speed will be slower due to one more step than the single-stage target detection algorithm. The target detection algorithm is highly targeted, and the model needs to be improved to adapt to the current task in the face of different recognition targets. At present, the immature fruits of most inventions are very different from the background color. For immature green fruits in the growth period, they are mostly similar in color to the background environment, and there will be situations such as leaves blocking and overlapping fruits, which increases the difficulty of target recognition. In terms of yield measurement, most inventions identify and count static images of immature fruits, while there are fewer inventions for the recognition and counting of dynamic fruits. Summary of the invention

[0003] In view of this, the present invention proposes a fruit counting method based on an improved coati optimization algorithm to achieve dynamic identification and counting to meet the actual needs of yield measurement tasks. The method comprises the following steps:

[0004] Obtain training images, perform data augmentation on the training images and construct a training set;

[0005] Based on the YOLOv7-tiny model, CBAM attention mechanism modules are added to the first branch and the second branch of the ELAN-T module in front of the P3 detection head to build an object detection model.

[0006] Introducing chaotic functions into the coati optimization algorithm;

[0007] The target detection model is trained based on the training set, WIoUv3 is used as the loss function, and the hyperparameters of the target detection model are optimized by combining the improved coati optimization algorithm to obtain the final target detection model.

[0008] The final target detection model and target tracking algorithm are combined to process the video to be tested and complete the tracking and counting tasks in real time.

[0009] In some embodiments, the process of acquiring training images, performing data enhancement on the training images and constructing a training set specifically includes:

[0010] Acquire fruit images in different environments, including sunny environments, cloudy environments, and scenes with leaves blocking them;

[0011] The collected fruit images are supplemented by adding noise, image inversion, vertical mirroring and horizontal mirroring;

[0012] The supplemented image set is divided into training set and test set.

[0013] In some embodiments, the process of optimizing the hyperparameters of the target detection model in combination with the improved coati optimization algorithm specifically includes:

[0014] Set relevant parameters;

[0015] Generate chaotic sequence based on chaotic mapping function and initialize relevant parameters;

[0016] Use individuals to perform global and local searches. At the same time, use the Levy flight strategy to further expand the search range and use the greedy strategy comparison to update the position.

[0017] After the position is updated, the optimal solution in the current iteration is compared with the historical optimal solution. If the optimal solution in the current iteration is greater than the historical optimal solution, the optimal solution in the current iteration is updated to the historical optimal solution; otherwise, the historical optimal solution remains unchanged.

[0018] It is determined that the termination condition is met, and the optimal hyperparameters are fixed according to the historical optimal solution to obtain the final target detection model.

[0019] In some embodiments, the process of combining the final target detection model and the target tracking algorithm to process the video to be detected and completing the tracking and counting task in real time specifically includes:

[0020] Based on the SORT algorithm, the re-identification count is introduced to build the StrongSORT target tracking algorithm;

[0021] The final target detection model is combined with the introduction of re-identification counting to construct the StrongSORT target tracking algorithm to track and count citrus fruits in the video.

[0022] The present invention also proposes a fruit counting system based on an improved coati optimization algorithm, the system comprising:

[0023] Image acquisition module, used to acquire images and build training sets;

[0024] Model building module, based on YOLOv7 -tiny Basic framework, introduce CBAM attention mechanism module, and build target detection model;

[0025] The training optimization module trains the target detection model based on the training set to obtain the WIoU v3is the loss function, and the target detection model is optimized by combining the chaotic function and the coati optimization algorithm to obtain the final target detection model;

[0026] The tracking and counting module is used to process the video to be tested in combination with the final target detection model and the target tracking algorithm to complete the tracking and counting task.

[0027] Based on the above scheme, the present invention provides a fruit counting method and system based on the improved coati optimization algorithm, taking YOLOv7-tiny as the benchmark model, and adding the CBAM attention mechanism in the ELAN-T module in front of the P3 detection head of YOLOv7-tiny to improve the model's feature extraction ability for small targets and citrus green fruits in complex backgrounds. By using the WIoU v3 loss function, the harmful gradients of low-quality instances generated by the overlap of citrus green fruits and the occlusion of leaves are reduced, and the detection ability of citrus green fruits in complex backgrounds and overlaps is further improved. The StrongSORT target tracking algorithm is used, and this algorithm is combined with the target detection model to form a citrus green fruit target detection and target tracking algorithm, which identifies, tracks and counts the video stream of citrus green fruits in the citrus orchard, providing technical support for the citrus orchard yield measurement task to meet the actual needs of the yield measurement task. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the steps of a fruit counting method based on an improved coati optimization algorithm of the present invention;

[0029] Figure 2 It is a schematic diagram of the structure of the improved ELAN-TC module;

[0030] Figure 3 The present invention is a structural block diagram of a fruit counting system based on an improved coati optimization algorithm. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] It should be noted that, for the convenience of description, only the parts related to the relevant invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0033] It should be understood that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0034] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0035] In the description of the embodiments of the present application, "plurality" means two or more than two. The following terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0036] In addition, flow charts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0037] Reference Figure 1 , is a flow chart of an optional example of a fruit counting method based on an improved coati optimization algorithm proposed in the present invention. The method can be applied to a computer device. The fruit counting method proposed in this embodiment may include but is not limited to the following steps:

[0038] Step S1, acquiring images and constructing a training set;

[0039] Step S2: Based on the YOLOv7-tiny basic framework, the CBAM attention mechanism module is introduced to build a target detection model;

[0040] Step S3, training the target detection model based on the training set, taking WIoU v3 as the loss function, and optimizing the hyperparameters of the target detection model in combination with the chaotic function and the coati optimization algorithm to obtain the final target detection model;

[0041] Step S4: Process the video to be tested in combination with the final target detection model and the target tracking algorithm to complete the tracking and counting task.

[0042] In some feasible embodiments, the step S1 specifically includes:

[0043] Handheld image acquisition equipment was used to obtain citrus fruit images in sunny and cloudy environments, with leaves blocking, clear and blurred scenes;

[0044] The collected citrus fruit images are augmented by adding salt and pepper noise, image inversion, vertical mirroring, horizontal mirroring and other data enhancement methods to expand the collected citrus fruit data. Finally, the original images and the data-enhanced images are used to construct a citrus fruit dataset.

[0045] The Labelimg tool was used to annotate the citrus fruit dataset, and the annotated data was divided into a training set and a validation set in a ratio of 8:2.

[0046] In some feasible embodiments, the step S2 specifically includes:

[0047] Taking YOLOv7-tiny as the benchmark model, a citrus fruit object detection model YOLO-CW is proposed.

[0048] The CBAM attention mechanism modules are added to the first branch and the second branch of the ELAN-T module in front of the P3 detection head respectively, and they are designed as ELAN-TC modules. This improves the feature extraction capability of small targets and citrus fruits in complex backgrounds, and reduces the computational complexity and parameter amount of the model.

[0049] In order to further extract features in the deep layer, a CBAM module is added in the first branch after channel and spatial convolution.

[0050] Since a single channel convolution retains less shallow feature information, it is difficult to extract the features of small targets such as citrus fruits in complex backgrounds. Therefore, in the second branch, a CBAM module is added after the channel convolution.

[0051] Due to the addition of the CBAM module, the calculation amount and parameter amount of the ELAN-TC module are reduced compared to the ELAN-T module. Figure 2 .

[0052] The CBAM attention mechanism module includes the channel attention module (ChannelAttention Module, CAM) and the spatial attention module (SpartialAttention Module, SAM), and the two modules are independent of each other. The first is the channel attention module.

[0053] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))) #(1)

[0054] Among them, F is the feature map input by the channel attention module, MLP is a two-layer neural network, and M c The feature map first passes through the channel attention module, performs global maximum pooling and global average pooling for width and height respectively, and then enters the MLP respectively. The output of the MLP is subjected to element-wise addition and sigmoid activation operation, and finally outputs M c .

[0055] M c Perform element-wise multiplication with the original feature map as the input F' of the spatial attention mechanism.

[0056] M s (F') = σ(f 7×7 ([MaxPool(F′),AvgPool(F′)])) #(2)

[0057] Among them, F' is the feature map input to the spatial attention module, f 7×7 is a 7×7 convolution kernel, M s is the final output of the spatial attention module. F' performs global maximum pooling and global average pooling of the channel, then performs channel concatenation, and then performs dimensionality reduction and sigmoid activation operation to output M s , and finally perform element-wise multiplication with F to complete the whole process. Since the CBAM module extracts the features of the target during channel convolution and spatial convolution, it discards useless feature information, thereby reducing the amount of calculation and parameters caused by useless information.

[0058] In some embodiments, the loss function in step S3 is expressed as follows:

[0059]

[0060] Among them, x and y represent the position of the center point of the prediction box, x gt and gt Represents the position of the center point of the calibration box, W g and H g Represents the width and height of the calibration box, is the overlap between the predicted box and the real box in the target detection task, is the monotonic focusing coefficient.

[0061] In the process of labeling citrus fruit samples, problems such as fruit overlap and leaf occlusion will make the labeled citrus fruit incomplete. Although the CIoU used in YOLOv7-tiny takes into account the overlapping area, center point distance, and aspect ratio of bounding box regression, the difference in aspect ratio is not the true difference between width and height and its confidence. This leads to the model predicting overlapping or occluded fruits and background as citrus fruits during training, which will aggravate such low-quality examples and reduce the generalization ability of the model. Therefore, in order to solve the problem that low-quality examples generated during training affect the training effect, the loss function CIoU is changed to WIoU (Wise IoU). WIoU v3 is further selected as the loss function.

[0062] Therefore, in the present invention, since citrus green fruits may have overlapping fruits and leaves blocking them, the calibration frames of these samples contain less effective parts of citrus green fruits during the data annotation process, and prediction frames will appear during the training process to calibrate these samples with the background. This makes these low-quality examples constantly affect the training process, and then affect the generalization and detection capabilities of the entire model. In the present invention, WIoU v3 is used to solve this problem. For citrus green fruits with overlapping fruits and leaves blocking them, the gradient gain of low-quality examples will be continuously adjusted during the training process, thereby effectively preventing harmful gradients generated by low-quality examples, so that the model focuses on ordinary examples and high-quality examples.

[0063] In some feasible embodiments, the hyperparameter optimization process in step S3 specifically includes:

[0064] Improve YOLO-CW hyperparameters using the modified Coati Smart Optimization Algorithm (TCOA).

[0065] Inspired by the hunting behavior of coatis, the algorithm mainly includes the hunting and attack strategies of coatis on iguanas and the strategies of escaping from predators. First, half of the raccoons climb a tree to hunt iguanas, while the other half gather under the tree and wait for the iguanas to land. When the iguanas land, the raccoons will hunt them.

[0066] For the raccoon in the tree.

[0067]

[0068] Where I is a random integer between 1 and 2, N is the number of raccoon populations, and x is the number of i (j) represents the current i-th raccoon individual, x best (j) represents the optimal position of the current raccoon population.

[0069] For iguanas and raccoons on the ground.

[0070] Iguana ground (j) = lbj +randowm·(ub j -lb j ) #(8)

[0071]

[0072] Among them j , lb j is the upper and lower limits of the current hyperparameters, Iguana ground (j) is the current position of the iguana, which represents the global optimal solution. ground ) and fitness(x i ) represents the fitness value of the iguana and the i-th raccoon individual, x i (j) is the jth hyperparameter of the i-th raccoon individual. Since the individual initialization process of the coati optimization algorithm is random and easy to lose population diversity, the use of a chaotic mapping function to generate a chaotic sequence for initialization can make the population distribution more uniform and improve the search efficiency. The present invention uses the TENT chaotic function to redefine:

[0073] X i :x i,j =ub j +(ub j -lb j )·z i #(10)

[0074] The TENT function expression is as follows, α=0.5.

[0075]

[0076] Among them, X i represents the hyperparameter group of the ith individual, x i,j represents the jth hyperparameter in the ith individual, lb j represents the lower limit of the jth hyperparameter in the ith individual, ub j represents the upper limit of the jth hyperparameter in the ith individual, z i Represents the i-th mapping vector in the TENT sequence.

[0077] During the hunting process of coatis, it is easy to fall into the local optimal solution. The Levy flight strategy is used to update the position, allowing some coatis individuals to move to a wider search space.

[0078]

[0079] Where S is the optimal solution of the coati, θ and ω follow the normal distribution, σ u With σ vis the variance of the normal distribution of θ and ω, and the parameter β is a random number in [0,2].

[0080] To escape from a predator, coatis will flee their original location and seek a safe location.

[0081]

[0082] Where t represents the current number of iterations, using the upper and lower limits of the current hyperparameters ub j , lb j Divide by t to get and Formula (14) is used to calculate the position of the current coati individual after escaping from the predator. Each coati in the population uses the greedy strategy to update its position after moving.

[0083] Set relevant parameters, such as population size, number of iterations, upper and lower limits; initialize the initial position of each individual, and save the initial historical optimal solution X g,best ;

[0084] Each individual is searched globally using formula (7) and formula (9), and locally using formula (13) and formula (14). Then formula (12) is used to further expand the search range to prevent the individual from entering the local optimum. Finally, the greedy strategy is used to compare and update the position. The value with the larger fitness value is selected for the final position update;

[0085] After the position is updated, select the optimal solution X in the current iteration best and the historical optimal solution X g,best For comparison, if X best >X g,best , then update the historical optimal solution, otherwise it remains unchanged.

[0086] After the termination condition is reached, the optimized optimal hyperparameter X g,best Train the model to get the final model.

[0087] In some embodiments, the step S4 specifically includes:

[0088] Build the StrongSORT target tracking algorithm;

[0089] The SORT target tracking algorithm combines a target detector and a tracker. The target detector is a trained target detection model, while the tracker is a combination of the Kalman filter and the Hungarian algorithm. To achieve the target tracking effect, the Kalman filter predicts the position and state of the next frame of target movement, while the Hungarian algorithm matches and associates the predicted position and state with the actual detection results. The StrongSORT algorithm is improved on the basis of the SORT algorithm and introduces re-identification technology. While paying attention to the target movement, it uses the model to extract the appearance features of the target and compares them with the features stored in the previous frame to determine whether it is the same object, thereby improving the tracking accuracy.

[0090] The YOLO-CW model is combined with the StrongSORT algorithm to track and count citrus fruits in the video.

[0091] Based on the above scheme, in order to verify the counting effect of the tracking counting algorithm, this study conducted manual counting and detection counting of the number of fruits, occlusion of leaves and complex backgrounds in the citrus orchard, and calculated the number of errors between manual counting and detection counting, and the results are shown in Table 1. It can be seen from the table that in the case of obvious fruits, less occlusion, severe occlusion and complex background, the error between manual counting and detection counting is within 3, which shows that the counting ability of the algorithm is not much different from that of manual counting, and meets the requirements for counting citrus green fruits in the citrus orchard.

[0092] Table 1. Related detection and counting data of citrus green fruit by manual and algorithm

[0093] Actual quantity / piece Model count / unit Number of errors / pieces 9 8 1 16 15 1 16 18 2 35 34 1

[0094] like Figure 3 As shown, a fruit counting imaging system based on an improved coati optimization algorithm comprises:

[0095] Image acquisition module, used to acquire images and build training sets;

[0096] The model building module is based on the YOLOv7-tiny basic framework, introduces the CBAM attention mechanism module, and builds a target detection model;

[0097] A training optimization module is used to train the target detection model based on the training set, and to optimize the hyperparameters of the target detection model by combining the chaotic function and the coati optimization algorithm with WIoU v3 as the loss function to obtain the final target detection model;

[0098] The tracking and counting module is used to process the video to be tested in combination with the final target detection model and the target tracking algorithm to complete the tracking and counting task.

[0099] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0100] A fruit counting imaging device based on an improved coati optimization algorithm:

[0101] at least one processor;

[0102] at least one memory for storing at least one program;

[0103] When the at least one program is executed by the at least one processor, the at least one processor implements the fruit counting imaging method based on the improved coati optimization algorithm as described above.

[0104] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0105] A storage medium stores instructions executable by a processor, wherein the instructions executable by the processor are used to implement the above-mentioned fruit counting imaging method based on the improved coati optimization algorithm when executed by the processor.

[0106] The contents of the above method embodiments are all applicable to the present storage medium embodiments. The functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0107] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A fruit counting method based on an improved coati optimization algorithm, characterized in that: The following steps are involved: Get images and build a training set; Based on the YOLOv7-tiny basic framework, the CBAM attention mechanism module is introduced to build a target detection model; The target detection model is trained based on the training set, WIoU v3 is used as the loss function, and the target detection model is hyper-parameter optimized by combining the chaotic function and the coati optimization algorithm to obtain the final target detection model; The final target detection model and the target tracking algorithm are combined to process the video to be tested to complete the tracking and counting task.

2. The fruit counting method based on the improved coati optimization algorithm according to claim 1, characterized in that: The step of introducing the CBAM attention mechanism module based on the YOLOv7-tiny basic framework and building a target detection model specifically includes: Based on the YOLOv7-tiny basic framework, CBAM attention mechanism modules are added to the first branch and the second branch of the ELAN-T module in front of the P3 detection head to build a target detection model; The CBAM attention mechanism module includes a channel attention module and a spatial attention module.

3. The fruit counting method based on the improved coati optimization algorithm according to claim 2, characterized in that: The formula of WIoU v3 loss function is as follows: Among them, x and y represent the position of the center point of the prediction box, x gt and gt Represents the position of the center point of the calibration box, W g and H g Represents the width and height of the calibration box, is the overlap between the predicted box and the real box in the target detection task, is the monotonic focusing coefficient.

4. The fruit counting method based on the improved coati optimization algorithm according to claim 2, characterized in that: The step of optimizing the hyperparameters of the target detection model by combining the chaotic function and the coati optimization algorithm specifically includes: Set relevant parameters; Generate a chaotic sequence based on a chaotic mapping function, and initialize the relevant parameters; Use individuals to perform global and local searches and update positions; Select the optimal solution in the current iteration and compare it with the historical optimal solution, and determine the historical optimal solution based on the comparison result; It is determined that the termination condition is met, and the optimal hyperparameters are fixed according to the historical optimal solution to obtain the final target detection model.

5. The fruit counting method based on the improved coati optimization algorithm according to claim 4, characterized in that: The initialization formula is as follows: X i :x i,j =ub j +(ub j -lb j )·z i Among them, X i represents the hyperparameter group of the ith individual, x i,j represents the jth hyperparameter in the ith individual, lb j represents the lower limit of the jth hyperparameter in the ith individual, ub j represents the upper limit of the jth hyperparameter in the ith individual, z i represents the i-th mapping vector in the TENT sequence, and α is a preset parameter.

6. The fruit counting method based on the improved coati optimization algorithm according to claim 5, characterized in that: in: The global search formula is as follows: Iguana ground (j)=lb j +random·(ub j -lb j ) Among them, Iguana ground (j) is the current position of the iguana, fitness(Iguana ground ) and fitness(x i ) represents the fitness value of the iguana and the i-th individual, x i (j) is the jth hyperparameter of the i-th individual; The formula for local search is as follows: i=1,2,…,N Where t represents the current iteration number.

7. The fruit counting method based on the improved coati optimization algorithm according to claim 6, characterized in that: During the search process, also include: Expand your search using the following formula: Where S is the optimal solution of the coati, θ and ω follow the normal distribution, σ u With σ v They represent the variance of the θ and ω normal distributions respectively, and the parameter β is a random number in [0,2].

8. A fruit counting system based on an improved coati optimization algorithm, characterized in that: include: Image acquisition module, used to acquire images and build training sets; The model building module is based on the YOLOv7-tiny basic framework, introduces the CBAM attention mechanism module, and builds a target detection model; A training optimization module is used to train the target detection model based on the training set, and to optimize the hyperparameters of the target detection model by combining the chaotic function and the coati optimization algorithm with WIoU v3 as the loss function to obtain the final target detection model; The tracking and counting module is used to process the video to be tested in combination with the final target detection model and the target tracking algorithm to complete the tracking and counting task.

9. A fruit counting device based on an improved coati optimization algorithm, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the fruit counting method based on the improved coati optimization algorithm as described in any one of claims 1 to 7.

Images