Improved YOLOv7-tiny-based strawberry flower and fruit detection method and system

Abstract

The invention discloses a strawberry flower and fruit detection method and system based on improved YOLOv7-tin, and relates to the technical field of computer vision inspection.The strawberry flower and fruit detection method comprises the steps that strawberry images of different growth cycles are obtained, and a strawberry image data set is established; performing data enhancement operation on the strawberry image data set to obtain a strawberry image enhancement data set; inputting the strawberry image enhancement data set into an improved YOLOv7-tiny strawberry detection model, and setting a loss function for training to obtain a trained strawberry detection model; and obtaining a strawberry image to be detected, inputting the strawberry image to the trained strawberry detection model, and obtaining a flower and fruit detection target of the strawberry image to be detected. According to the method, the flower and fruit targets of the strawberries in each period can be accurately detected in a complex actual scene, the calculation amount is reduced, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision detection, and more specifically, to a strawberry flower and fruit detection method and system based on improved YOLOv7-tiny. Background Art

[0002] Currently, the identification of strawberry flowers and fruits is mainly done by manpower, which faces problems such as long cycle, high frequency, high labor intensity and high cost, thus greatly restricting the efficiency of strawberry production. However, with the development of intelligent agriculture, many studies on machine learning and deep network technology based on computer vision have been applied to strawberry detection, and accurately identifying strawberries through algorithms has become a research hotspot.

[0003] Machine learning has a long history of application in the field of strawberry production. When the loss of the outline information of ripe strawberries is less than 1 / 2, the existing machine learning technology has an average relative deviation of 4.8%, whether it is a single separated ripe strawberry or a covered, overlapping or close-together ripe strawberry, which can meet the target recognition accuracy requirements of the strawberry picking robot. However, machine learning can only complete the recognition of strawberry fruits in relatively simple scenarios, and cannot perform detection and positioning. In actual production complex scenarios, the accuracy is low and cannot meet actual production requirements. In recent years, with the development of deep learning, researchers have used deep learning to replace machine learning to develop strawberry fruit detection methods. However, these studies are mainly based on the detection of ripe strawberry fruits, lacking methods for simultaneous detection of strawberry flowers and fruits, and the occlusion problem in actual complex growth scenarios is more severe, resulting in inaccurate detection results. Summary of the invention

[0004] One of the purposes of the present invention is to provide a strawberry flower and fruit detection method based on improved YOLOv7-tiny, so as to overcome the defect that the above-mentioned prior art cannot accurately detect strawberry flowers and fruits at the same time in actual complex growth scenarios; the second purpose is to provide a strawberry flower and fruit detection system based on improved YOLOv7-tin; the third purpose is to provide a computer device.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] The present invention provides a strawberry flower and fruit detection method based on improved YOLOv7-tiny, comprising:

[0007] Obtain strawberry images at different growth periods and build a strawberry image dataset;

[0008] Performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset;

[0009] Input the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, set the loss function for training, and obtain a trained strawberry detection model;

[0010] Get the strawberry image to be detected, input it into the trained strawberry detection model, and obtain the flower and fruit detection target of the strawberry image to be detected.

[0011] It can be understood that the strawberry image dataset includes strawberry images taken from multiple angles and directions in all growth cycles. These strawberry images are manually divided into detection targets and annotated with the real labels of strawberry flowers and / or strawberry fruits. They are used together in the subsequent training process of the strawberry detection model, so that the strawberry detection model can learn more complex strawberry flower and fruit characteristics.

[0012] Preferably, the data enhancement operation includes one or more of brightness adjustment, motion blur, adding Gaussian noise, multi-angle rotation, and mirroring.

[0013] It should be noted that in the actual growth detection of strawberries, there will be various interferences during the shooting process, resulting in imbalanced collected strawberry image data. Before training the strawberry detection model, the above offline data enhancement operation can be used to reduce overfitting, stimulate the potential of the strawberry detection model, and improve the generalization performance of the strawberry detection model.

[0014] Preferably, the improved YOLOv7-tiny strawberry detection model includes a stem network unit, a trunk network unit and a detection head network unit connected in sequence.

[0015] Preferably, the stem network unit comprises an FFS module;

[0016] The FFS module has a first CBS block, a second CBS block, a third CBS block, a fourth CBS block, a fifth CBS block, a sixth CBS block, a seventh CBS block and a first connection layer;

[0017] The first CBS block, the second CBS block, the third CBS block, the first connection layer, the fourth CBS block, the fifth CBS block, and the sixth CBS block are connected in sequence;

[0018] The output end of the first CBS block is also connected to the input end of the seventh CBS block, and the output end of the seventh CBS block is also connected to the input end of the first connection layer;

[0019] The output end of the sixth CBS block is connected to the input end of the backbone network unit.

[0020] It should be noted that the stem network unit in the existing YOLOv7 neural network often uses the stem module for downsampling operations, which will cause the loss of the original features of the input image. However, using the FFS module for downsampling operations can effectively reduce the loss of original features and ensure the accuracy of the detection results.

[0021] Preferably, the backbone network unit comprises a first HG module, an eighth CBS block, a second HG module, a ninth CBS block, a third HG module, a tenth CBS block and a fourth HG module connected in sequence;

[0022] The input end of the first HG module is connected to the output end of the sixth CBS block, and the output end of the second HG module, the output end of the third HG module, and the output end of the fourth HG module are all connected to the input end of the detection head network unit;

[0023] The first HG module, the second HG module, the third HG module and the fourth HG module have the same structure, and all include the eleventh CBS block, the twelfth CBS block, the thirteenth CBS block, the fourteenth CBS block, the fifteenth CBS block, the sixteenth CBS block, the second connection layer, the seventeenth CBS block and the eighteenth CBS block which are connected in sequence; the input end of the eleventh CBS block, the output end of the eleventh CBS block, the output end of the twelfth CBS block, the output end of the thirteenth CBS block, the output end of the fourteenth CBS block and the output end of the fifteenth CBS block are also connected to the input end of the second connection layer.

[0024] It should be noted that the structure of the HG module uses more multi-level splicing operations to expand the feature information, which can better detect various forms of strawberries in complex growth scenes and multiple growth cycles, and improve the detection performance of the detection model.

[0025] Preferably, the detection head network unit includes a nineteenth CBS block, a twentieth CBS block, a SPPCSPC module, a first UP module, a third connection layer, a first Elan-t module, a second UP module, a fourth connection layer, a second Elan-t module, a fifth connection layer, a twenty-first CBS block, a third Elan-t module, a sixth connection layer, a twenty-second CBS block, a fourth Elan-t module, a first Rep module, a first detection layer, a second Rep module, a second detection layer, a third Rep module and a third detection layer;

[0026] The input end of the SPPCSPC module is connected to the output end of the fourth HG module, the first output end of the SPPCSPC module is connected to the input end of the first UP module, the output end of the first UP module is connected to the first input end of the third connection layer, the output end of the third connection layer is connected to the input end of the first Elan-t module, the first output end of the first Elan-t module is connected to the input end of the second UP module, the output end of the second UP module is connected to the first input end of the fourth connection layer, and the output end of the fourth connection layer is connected to the input end of the second Elan-t module;

[0027] The input end of the nineteenth CBS block is connected to the output end of the third HG module, and the output end of the nineteenth CBS block is connected to the second input end of the third connection layer; the input end of the twentieth CBS block is connected to the output end of the second HG module, and the output end of the twentieth CBS block is connected to the second input end of the fourth connection layer;

[0028] The first output end of the second Elan-t module is connected to the input end of the third Rep module, and the output end of the third Rep module is connected to the input end of the third detection layer;

[0029] The second output end of the second Elan-t module is connected to the input end of the twenty-first CBS block, the output end of the twenty-first CBS block is connected to the first input end of the sixth connection layer, the output end of the sixth connection layer is connected to the input end of the third Elan-t module, the first output end of the third Elan-t module is connected to the input end of the second Rep module, the output end of the second Rep module is connected to the input end of the second detection layer; the second output end of the first Elan-t module is connected to the second input end of the sixth connection layer;

[0030] The second output end of the third Elan-t module is connected to the input end of the twenty-second CBS block, the output end of the twenty-second CBS block is connected to the first input end of the fifth connection layer, the output end of the fifth connection layer is connected to the input end of the fourth Elan-t module, the output end of the fourth Elan-t module is connected to the input end of the first Rep module, and the output end of the first Rep module is connected to the input end of the first detection layer;

[0031] The second input terminal of the SPPCSPC module is connected to the second input terminal of the fifth connection layer.

[0032] Preferably, the SPPCSPC module includes a twenty-third CBS block, a twenty-fourth CBS block, a twenty-fifth CBS block, a twenty-sixth CBS block, a first MP block, a second MP block, a third MP block, a seventh connection layer and an eighth connection layer;

[0033] The 23rd CBS block and the 26th CBS block are arranged in parallel, the output end of the 23rd CBS block is respectively connected to the input ends of the first MP block, the second MP block, the third MP block and the seventh connection layer, the output ends of the first MP block, the second MP block and the third MP block are also connected to the input end of the seventh connection layer, the output end of the seventh connection layer is connected to the input end of the 24th CBS block, the output end of the 24th CBS block is connected to the input end of the eighth connection layer, the output end of the eighth connection layer is connected to the input end of the 25th CBS block; the output end of the 26th CBS block is connected to the input end of the eighth connection layer.

[0034] Preferably, the first Elan-t module, the second Elan-t module, the third Elan-t module, and the fourth Elan-t module have the same structure, and all include a twenty-seventh CBS block, a twenty-eighth CBS block, a twenty-ninth CBS block, a thirtieth CBS block, a thirty-first CBS block, and a ninth connection layer;

[0035] The 27th CBS block and the 31st CBS block are arranged in parallel, the output end of the 27th CBS block is connected to the input end of the 28th CBS block, the output end of the 28th CBS block is connected to the input end of the 29th CBS block, the output end of the 29th CBS block is connected to the input end of the ninth connection layer, and the output end of the ninth connection layer is connected to the input end of the 30th CBS block;

[0036] The output ends of the 27th CBS block, the 28th CBS block and the 31st CBS block are all connected to the input end of the ninth connection layer.

[0037] Preferably, the first UP module and the second UP module have the same structure, and both include a thirty-second CBS block and an upsampling layer connected in sequence.

[0038] The present invention also provides a strawberry flower and fruit detection system based on improved YOLOv7-tiny, which is used to implement the above detection method, including:

[0039] An image acquisition module is used to acquire strawberry images of different growth periods and establish a strawberry image dataset;

[0040] A data enhancement module, used for performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset;

[0041] A model training module is used to input the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, set a loss function for training, and obtain a trained strawberry detection model;

[0042] The flower and fruit detection module is used to obtain the strawberry image to be detected, input it into the trained strawberry detection model, and obtain the flower and fruit detection target of the strawberry image to be detected.

[0043] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0044] The present invention first obtains strawberry images of different growth cycles to establish a strawberry image data set, and performs a data enhancement operation to obtain a strawberry image enhancement data set; then the strawberry image enhancement data set is input into an improved YOLOv7-tiny strawberry detection model, a loss function is set for training, and a trained strawberry detection model is obtained; finally, the trained strawberry detection model is used to detect the strawberry image to be detected, and the flower and fruit detection target of the strawberry image to be detected is obtained. The strawberry image data set of the present invention includes strawberry images shot at multiple angles and multiple directions in all growth cycles, and then combined with the data enhancement operation, the training data of the subsequent training process is sufficient, the overfitting phenomenon of the model is reduced, and the generalization performance of the model is improved; the improved YOLOv7-tiny strawberry detection model can effectively reduce the loss of original features in the image feature extraction process, and the feature information is expanded by using a multi-level splicing operation, so that the flower and fruit targets of strawberries in various periods can be accurately detected in complex actual scenes, the amount of calculation is reduced, and the detection efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a strawberry flower and fruit detection method based on improved YOLOv7-tiny described in Example 1;

[0046] Figure 2 It is a schematic diagram of the structure of the strawberry detection model of the improved YOLOv7-tiny described in Example 2;

[0047] Figure 3 This is a schematic structural diagram of the FFS module described in Example 2;

[0048] Figure 4 This is a schematic diagram of the structure of the HG module described in Example 2;

[0049] Figure 5 It is a structural schematic diagram of the SPPCSPC module described in Example 2;

[0050] Figure 6 This is a schematic diagram of the structure of the Elan-t module described in Example 2;

[0051] Figure 7 This is a schematic diagram of the structure of the UP module described in Example 2;

[0052] Figure 8A schematic diagram of the structures of the first CBS block, the second CBS block, the third CBS block, the fourth CBS block, the eighth CBS block, the ninth CBS block, the tenth CBS block, the eleventh CBS block, the twelfth CBS block, the thirteenth CBS block, the fourteenth CBS block, the fifteenth CBS block, the sixteenth CBS block, the twenty-first CBS block, the twenty-second CBS block, the twenty-eighth CBS block, and the twenty-ninth CBS block described in Example 2;

[0053] Fig. 9 A schematic diagram of the structures of the fifth CBS block, the sixth CBS block, the seventh CBS block, the seventeenth CBS block, the eighteenth CBS block, the nineteenth CBS block, the twentieth CBS block, the twenty-third CBS block, the twenty-fourth CBS block, the twenty-fifth CBS block, the twenty-sixth CBS block, the twenty-seventh CBS block, the thirtieth CBS block, the thirty-first CBS block, and the thirty-second CBS block described in Example 2;

[0054] Fig.10 This is a schematic diagram of the structure of a strawberry flower and fruit detection system based on improved YOLOv7-tiny as described in Example 3. DETAILED DESCRIPTION

[0055] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0056] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0057] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0058] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0059] Example 1

[0060] This embodiment provides a strawberry flower and fruit detection method based on improved YOLOv7-tiny. Figure 1 As shown, including:

[0061] S1: Obtain strawberry images in different growth periods and establish a strawberry image dataset;

[0062] S2: performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset;

[0063] S3: inputting the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, setting a loss function for training, and obtaining a trained strawberry detection model;

[0064] S4: Obtain a strawberry image to be detected, input it into a trained strawberry detection model, and obtain a flower and fruit detection target of the strawberry image to be detected.

[0065] In the specific implementation process, the present implementation first obtains strawberry images of different growth cycles to establish a strawberry image data set, and performs data enhancement operation to obtain a strawberry image enhancement data set; then the strawberry image enhancement data set is input into the strawberry detection model of the improved YOLOv7-tiny, and the loss function is set for training to obtain a trained strawberry detection model; finally, the trained strawberry detection model is used to detect the strawberry image to be detected, and the flower and fruit detection target of the strawberry image to be detected is obtained. The strawberry image data set of the present embodiment includes strawberry images taken from multiple angles and multiple directions of all growth cycles, and then combined with the data enhancement operation, the training data of the subsequent training process is sufficient, the overfitting phenomenon of the model is reduced, and the generalization performance of the model is improved; the strawberry detection model of the improved YOLOv7-tiny can effectively reduce the loss of original features in the image feature extraction process, and the feature information is expanded by using a multi-level splicing operation, so that the flower and fruit targets of strawberries in various periods can be accurately detected in complex actual scenes, which reduces the amount of calculation and improves the detection efficiency.

[0066] Example 2

[0067] This embodiment provides a strawberry flower and fruit detection method based on improved YOLOv7-tiny, including:

[0068] S1: Obtain strawberry images in different growth periods and establish a strawberry image dataset;

[0069] It can be understood that the strawberry image dataset includes strawberry images taken from multiple angles and directions in all growth cycles. These strawberry images are manually divided into detection targets and annotated with the real labels of strawberry flowers and / or strawberry fruits. They are used together in the subsequent training process of the strawberry detection model, so that the strawberry detection model can learn more complex strawberry flower and fruit characteristics.

[0070] S2: performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset;

[0071] The data enhancement operation includes one or more of brightness adjustment, motion blur, adding Gaussian noise, multi-angle rotation, and mirroring.

[0072] It should be noted that in the actual growth detection of strawberries, there will be various interferences during the shooting process, resulting in imbalanced collected strawberry image data. Before training the strawberry detection model, the above offline data enhancement operation can be used to reduce overfitting, stimulate the potential of the strawberry detection model, and improve the generalization performance of the strawberry detection model.

[0073] S3: inputting the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, setting a loss function for training, and obtaining a trained strawberry detection model;

[0074] like Figure 2 As shown, the improved YOLOv7-tiny strawberry detection model includes a stem network unit, a trunk network unit and a detection head network unit connected in sequence;

[0075] The stem network unit includes an FFS module;

[0076] The backbone network unit includes a first HG module, an eighth CBS block, a second HG module, a ninth CBS block, a third HG module, a tenth CBS block and a fourth HG module connected in sequence;

[0077] The input end of the first HG module is connected to the output end of the FFS module, and the output end of the second HG module, the output end of the third HG module, and the output end of the fourth HG module are all connected to the input end of the detection head network unit;

[0078] The detection head network unit includes a nineteenth CBS block, a twentieth CBS block, an SPPCSPC module, a first UP module, a third connection layer, a first Elan-t module, a second UP module, a fourth connection layer, a second Elan-t module, a fifth connection layer, a twenty-first CBS block, a third Elan-t module, a sixth connection layer, a twenty-second CBS block, a fourth Elan-t module, a first Rep module, a first detection layer, a second Rep module, a second detection layer, a third Rep module and a third detection layer;

[0079] The input end of the SPPCSPC module is connected to the output end of the fourth HG module, the first output end of the SPPCSPC module is connected to the input end of the first UP module, the output end of the first UP module is connected to the first input end of the third connection layer, the output end of the third connection layer is connected to the input end of the first Elan-t module, the first output end of the first Elan-t module is connected to the input end of the second UP module, the output end of the second UP module is connected to the first input end of the fourth connection layer, and the output end of the fourth connection layer is connected to the input end of the second Elan-t module;

[0080] The input end of the nineteenth CBS block is connected to the output end of the third HG module, and the output end of the nineteenth CBS block is connected to the second input end of the third connection layer; the input end of the twentieth CBS block is connected to the output end of the second HG module, and the output end of the twentieth CBS block is connected to the second input end of the fourth connection layer;

[0081] The first output end of the second Elan-t module is connected to the input end of the third Rep module, and the output end of the third Rep module is connected to the input end of the third detection layer;

[0082] The second output end of the second Elan-t module is connected to the input end of the twenty-first CBS block, the output end of the twenty-first CBS block is connected to the first input end of the sixth connection layer, the output end of the sixth connection layer is connected to the input end of the third Elan-t module, the first output end of the third Elan-t module is connected to the input end of the second Rep module, the output end of the second Rep module is connected to the input end of the second detection layer; the second output end of the first Elan-t module is connected to the second input end of the sixth connection layer;

[0083] The second output end of the third Elan-t module is connected to the input end of the twenty-second CBS block, the output end of the twenty-second CBS block is connected to the first input end of the fifth connection layer, the output end of the fifth connection layer is connected to the input end of the fourth Elan-t module, the output end of the fourth Elan-t module is connected to the input end of the first Rep module, and the output end of the first Rep module is connected to the input end of the first detection layer;

[0084] The second input terminal of the SPPCSPC module is connected to the second input terminal of the fifth connection layer.

[0085] The SPPCSPC module includes a twenty-third CBS block, a twenty-fourth CBS block, a twenty-fifth CBS block, a twenty-sixth CBS block, a first MP block, a second MP block, a third MP block, a seventh connection layer and an eighth connection layer;

[0086] The 23rd CBS block and the 26th CBS block are arranged in parallel, the output end of the 23rd CBS block is respectively connected to the input ends of the first MP block, the second MP block, the third MP block and the seventh connection layer, the output ends of the first MP block, the second MP block and the third MP block are also connected to the input end of the seventh connection layer, the output end of the seventh connection layer is connected to the input end of the 24th CBS block, the output end of the 24th CBS block is connected to the input end of the eighth connection layer, the output end of the eighth connection layer is connected to the input end of the 25th CBS block; the output end of the 26th CBS block is connected to the input end of the eighth connection layer.

[0087] like Figure 3 As shown, the FFS module includes a first CBS block, a second CBS block, a third CBS block, a fourth CBS block, a fifth CBS block, a sixth CBS block, a seventh CBS block and a first connection layer;

[0088] The first CBS block, the second CBS block, the third CBS block, the first connection layer, the fourth CBS block, the fifth CBS block, and the sixth CBS block are connected in sequence;

[0089] The output end of the first CBS block is also connected to the input end of the seventh CBS block, and the output end of the seventh CBS block is also connected to the input end of the first connection layer.

[0090] It should be noted that the stem network unit in the existing YOLOv7 neural network often uses the stem module for downsampling operations, which will cause the loss of the original features of the input image. However, using the FFS module for downsampling operations can effectively reduce the loss of original features and ensure the accuracy of the detection results.

[0091] like Figure 4 As shown, the first HG module, the second HG module, the third HG module, and the fourth HG module have the same structure, and all include an eleventh CBS block, a twelfth CBS block, a thirteenth CBS block, a fourteenth CBS block, a fifteenth CBS block, a sixteenth CBS block, a second connection layer, a seventeenth CBS block, and an eighteenth CBS block connected in sequence; the input end of the eleventh CBS block, the output end of the eleventh CBS block, the output end of the twelfth CBS block, the output end of the thirteenth CBS block, the output end of the fourteenth CBS block, and the output end of the fifteenth CBS block are also connected to the input end of the second connection layer.

[0092] It should be noted that the structure of the HG module uses more multi-level splicing operations to expand the feature information, which can better detect various forms of strawberries in complex growth scenes and multiple growth cycles, and improve the detection performance of the detection model.

[0093] like Figure 5 As shown, the SPPCSPC module includes a sixteenth CBS block, a seventeenth CBS block, an eighteenth CBS block, a nineteenth CBS block, a first MP block, a second MP block, a third MP block, a seventh connection layer and an eighth connection layer;

[0094] The sixteenth CBS block and the nineteenth CBS block are arranged in parallel, the output end of the sixteenth CBS block is respectively connected to the input ends of the first MP block, the second MP block, the third MP block and the seventh connection layer, the output ends of the first MP block, the second MP block and the third MP block are also connected to the input end of the seventh connection layer, the output end of the seventh connection layer is connected to the input end of the seventeenth CBS block, the output end of the seventeenth CBS block is connected to the input end of the eighth connection layer, the output end of the eighth connection layer is connected to the input end of the eighteenth CBS block; the output end of the nineteenth CBS block is connected to the input end of the eighth connection layer.

[0095] like Figure 6As shown, the first Elan-t module, the second Elan-t module, the third Elan-t module, and the fourth Elan-t module have the same structure, and all include a twenty-seventh CBS block, a twenty-eighth CBS block, a twenty-ninth CBS block, a thirtieth CBS block, a thirty-first CBS block, and a ninth connection layer;

[0096] The 27th CBS block and the 31st CBS block are arranged in parallel, the output end of the 27th CBS block is connected to the input end of the 28th CBS block, the output end of the 28th CBS block is connected to the input end of the 29th CBS block, the output end of the 29th CBS block is connected to the input end of the ninth connection layer, and the output end of the ninth connection layer is connected to the input end of the 30th CBS block;

[0097] The output ends of the 27th CBS block, the 28th CBS block and the 31st CBS block are all connected to the input end of the ninth connection layer.

[0098] like Figure 7 As shown, the first UP module and the second UP module have the same structure, both including a thirty-second CBS block and an upsampling layer connected in sequence.

[0099] like Figure 8 As shown, the first CBS block, the second CBS block, the third CBS block, the fourth CBS block, the eighth CBS block, the ninth CBS block, the tenth CBS block, the eleventh CBS block, the twelfth CBS block, the thirteenth CBS block, the fourteenth CBS block, the fifteenth CBS block, the sixteenth CBS block, the twenty-first CBS block, the twenty-second CBS block, the twenty-eighth CBS block, and the twenty-ninth CBS block have the same structure, and all include a DSConv convolution layer, a first normalization layer, and a first SiLU activation function layer connected in sequence.

[0100] The application of DSConv convolution layer greatly reduces the amount of calculation and improves the efficiency of model detection. The difference between each CBS block is only the convolution kernel and step size. Among them, the convolution kernel size of the DSConv convolution layer in the first CBS block, the fourth CBS block, the eighth CBS block, the ninth CBS block, the tenth CBS block, the twenty-first CBS block, and the twenty-second CBS block is 3, and the step size is 2; the convolution kernel size of the DSConv convolution layer in the second CBS block, the third CBS block, the eleventh CBS block, the twelfth CBS block, the thirteenth CBS block, the fourteenth CBS block, the fifteenth CBS block, the sixteenth CBS block, the twenty-eighth CBS block, and the twenty-ninth CBS block is 3, and the step size is 1.

[0101] like Fig. 9As shown, the fifth CBS block, the sixth CBS block, the seventh CBS block, the seventeenth CBS block, the eighteenth CBS block, the nineteenth CBS block, the twentieth CBS block, the twenty-third CBS block, the twenty-fourth CBS block, the twenty-fifth CBS block, the twenty-sixth CBS block, the twenty-seventh CBS block, the thirtieth CBS block, the thirty-first CBS block, and the thirty-second CBS block have the same structure, and all include a convolutional layer, a second normalization layer, and a second SiLU activation function layer connected in sequence;

[0102] The convolution kernel size of the convolution layers in the fifth CBS block, the sixth CBS block, the seventh CBS block, the seventeenth CBS block, the eighteenth CBS block, the nineteenth CBS block, the twentieth CBS block, the twenty-third CBS block, the twenty-fourth CBS block, the twenty-fifth CBS block, the twenty-sixth CBS block, the twenty-seventh CBS block, the thirtieth CBS block, the thirty-first CBS block, and the thirty-second CBS block is 1, and the step size is 1.

[0103] It should be noted that the strawberry image enhancement dataset is input into the improved YOLOv7-tiny strawberry detection model, and the loss function, batch, learning rate and other hyperparameters are set to obtain the predicted label of each strawberry image; the loss function is the cross entropy loss function, and its calculation formula is as follows:

[0104]

[0105] In the formula, x i represents the i-th strawberry image, y i represents the predicted label of the i-th strawberry image, y i ′ represents the true label of the i-th strawberry image, and n represents the number of strawberry images; until the loss value of the loss function no longer decreases, a trained strawberry detection model with initialization parameters is obtained.

[0106] Afterwards, the F1 score and average precision are used to evaluate the accuracy of the trained strawberry detection model; the definition of the F1 score is as follows:

[0107]

[0108] Among them, P and R refer to precision and recall respectively;

[0109]

[0110] Among them, TP refers to the number of samples in any category where positive sample data is correctly identified; FN refers to the number of samples in any category where positive sample data is not correctly identified; FP refers to the number of samples in any category where non-positive sample data is identified as positive samples;

[0111] The mean average accuracy is defined as:

[0112]

[0113] Where: mAP is the average accuracy of all categories in the strawberry image enhancement dataset, and N represents the number of target categories in the strawberry image enhancement dataset.

[0114] S4: Obtain a strawberry image to be detected, input it into a trained strawberry detection model, and obtain a flower and fruit detection target of the strawberry image to be detected.

[0115] In actual use, we obtain strawberry images of different growth cycles and perform data enhancement operations to create a strawberry image enhancement dataset STB-Dataset, which contains 4603 actual growth images of strawberries taken at various angles and orientations. The images are divided into training set, validation set and test set, and the categories are divided into mature, immature and flowering stages. The number of images and category distribution are shown in the following table:

[0116] Number and category distribution Quantity / sheet Mature / Immature / Flowering period / piece Training set 3682 5075 10747 2716 Validation set 460 574 1181 295 Test Set 461 584 1358 304

[0117] On the strawberry image enhancement dataset STB-Dataset, the trained strawberry detection model provided in this embodiment is verified to be compared with the traditional YOLOv7-tiny benchmark model, YOLOv4 model, YOLOv5s model, and YOLOv8s model in terms of Flops, mAP@0.5, mAP@.5-.95, F1 score, and param. The comparison results are shown in the following table:

[0118]

[0119] As can be seen from the above table, the Flops of the trained strawberry detection model provided in this embodiment is only 5.7G, the mAP@0.5 reaches 94.1%, the mAP@.5-.95 reaches 70.1%, and the F1 score reaches 0.91, which are all better than other models in the comparative experiment.

[0120] Example 3

[0121] This embodiment provides a strawberry flower and fruit detection system based on improved YOLOv7-tiny, which is used to implement the detection method described in Example 1 or 2. Fig.10 As shown, including:

[0122] An image acquisition module is used to acquire strawberry images of different growth periods and establish a strawberry image dataset;

[0123] A data enhancement module, used for performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset;

[0124] A model training module is used to input the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, set a loss function for training, and obtain a trained strawberry detection model;

[0125] The flower and fruit detection module is used to obtain the strawberry image to be detected, input it into the trained strawberry detection model, and obtain the flower and fruit detection target of the strawberry image to be detected.

[0126] The same or similar reference numerals correspond to the same or similar components;

[0127] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0128] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A strawberry flower and fruit detection method based on improved YOLOv7-tiny, characterized in that: include: Obtain strawberry images at different growth periods and build a strawberry image dataset; Performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset; Input the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, set the loss function for training, and obtain a trained strawberry detection model; Get the strawberry image to be detected, input it into the trained strawberry detection model, and obtain the flower and fruit detection target of the strawberry image to be detected.

2. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 1, characterized in that, The data enhancement operation includes one or more of brightness adjustment, motion blur, adding Gaussian noise, multi-angle rotation, and mirroring.

3. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 1, characterized in that, The improved YOLOv7-tiny strawberry detection model includes a stem network unit, a trunk network unit and a detection head network unit connected in sequence.

4. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 3, characterized in that, The stem network unit includes an FFS module; The FFS module has a first CBS block, a second CBS block, a third CBS block, a fourth CBS block, a fifth CBS block, a sixth CBS block, a seventh CBS block and a first connection layer; The first CBS block, the second CBS block, the third CBS block, the first connection layer, the fourth CBS block, the fifth CBS block, and the sixth CBS block are connected in sequence; The output end of the first CBS block is also connected to the input end of the seventh CBS block, and the output end of the seventh CBS block is also connected to the input end of the first connection layer; The output end of the sixth CBS block is connected to the input end of the backbone network unit.

5. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 4, characterized in that, The backbone network unit includes a first HG module, an eighth CBS block, a second HG module, a ninth CBS block, a third HG module, a tenth CBS block and a fourth HG module connected in sequence; The input end of the first HG module is connected to the output end of the sixth CBS block, and the output end of the second HG module, the output end of the third HG module, and the output end of the fourth HG module are all connected to the input end of the detection head network unit; The first HG module, the second HG module, the third HG module and the fourth HG module have the same structure, and all include the eleventh CBS block, the twelfth CBS block, the thirteenth CBS block, the fourteenth CBS block, the fifteenth CBS block, the sixteenth CBS block, the second connection layer, the seventeenth CBS block and the eighteenth CBS block which are connected in sequence; the input end of the eleventh CBS block, the output end of the eleventh CBS block, the output end of the twelfth CBS block, the output end of the thirteenth CBS block, the output end of the fourteenth CBS block and the output end of the fifteenth CBS block are also connected to the input end of the second connection layer.

6. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 5, characterized in that, The detection head network unit includes a nineteenth CBS block, a twentieth CBS block, an SPPCSPC module, a first UP module, a third connection layer, a first Elan-t module, a second UP module, a fourth connection layer, a second Elan-t module, a fifth connection layer, a twenty-first CBS block, a third Elan-t module, a sixth connection layer, a twenty-second CBS block, a fourth Elan-t module, a first Rep module, a first detection layer, a second Rep module, a second detection layer, a third Rep module and a third detection layer; The input end of the SPPCSPC module is connected to the output end of the fourth HG module, the first output end of the SPPCSPC module is connected to the input end of the first UP module, the output end of the first UP module is connected to the first input end of the third connection layer, the output end of the third connection layer is connected to the input end of the first Elan-t module, the first output end of the first Elan-t module is connected to the input end of the second UP module, the output end of the second UP module is connected to the first input end of the fourth connection layer, and the output end of the fourth connection layer is connected to the input end of the second Elan-t module; The input end of the nineteenth CBS block is connected to the output end of the third HG module, and the output end of the nineteenth CBS block is connected to the second input end of the third connection layer; the input end of the twentieth CBS block is connected to the output end of the second HG module, and the output end of the twentieth CBS block is connected to the second input end of the fourth connection layer; The first output end of the second Elan-t module is connected to the input end of the third Rep module, and the output end of the third Rep module is connected to the input end of the third detection layer; The second output end of the second Elan-t module is connected to the input end of the twenty-first CBS block, the output end of the twenty-first CBS block is connected to the first input end of the sixth connection layer, the output end of the sixth connection layer is connected to the input end of the third Elan-t module, the first output end of the third Elan-t module is connected to the input end of the second Rep module, the output end of the second Rep module is connected to the input end of the second detection layer; the second output end of the first Elan-t module is connected to the second input end of the sixth connection layer; The second output end of the third Elan-t module is connected to the input end of the twenty-second CBS block, the output end of the twenty-second CBS block is connected to the first input end of the fifth connection layer, the output end of the fifth connection layer is connected to the input end of the fourth Elan-t module, the output end of the fourth Elan-t module is connected to the input end of the first Rep module, and the output end of the first Rep module is connected to the input end of the first detection layer; The second input terminal of the SPPCSPC module is connected to the second input terminal of the fifth connection layer.

7. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 6, characterized in that, The SPPCSPC module includes a twenty-third CBS block, a twenty-fourth CBS block, a twenty-fifth CBS block, a twenty-sixth CBS block, a first MP block, a second MP block, a third MP block, a seventh connection layer and an eighth connection layer; The 23rd CBS block and the 26th CBS block are arranged in parallel, the output end of the 23rd CBS block is respectively connected to the input ends of the first MP block, the second MP block, the third MP block and the seventh connection layer, the output ends of the first MP block, the second MP block and the third MP block are also connected to the input end of the seventh connection layer, the output end of the seventh connection layer is connected to the input end of the 24th CBS block, the output end of the 24th CBS block is connected to the input end of the eighth connection layer, the output end of the eighth connection layer is connected to the input end of the 25th CBS block; the output end of the 26th CBS block is connected to the input end of the eighth connection layer.

8. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 6, characterized in that, The first Elan-t module, the second Elan-t module, the third Elan-t module, and the fourth Elan-t module have the same structure, and all include a twenty-seventh CBS block, a twenty-eighth CBS block, a twenty-ninth CBS block, a thirtieth CBS block, a thirty-first CBS block, and a ninth connection layer; The 27th CBS block and the 31st CBS block are arranged in parallel, the output end of the 27th CBS block is connected to the input end of the 28th CBS block, the output end of the 28th CBS block is connected to the input end of the 29th CBS block, the output end of the 29th CBS block is connected to the input end of the ninth connection layer, and the output end of the ninth connection layer is connected to the input end of the 30th CBS block; The output ends of the 27th CBS block, the 28th CBS block and the 31st CBS block are all connected to the input end of the ninth connection layer.

9. The strawberry flower and fruit detection method based on improved YOLOv7-tiny according to claim 6, characterized in that, The first UP module and the second UP module have the same structure, both including a thirty-second CBS block and an upsampling layer connected in sequence.

10. A strawberry flower and fruit detection system based on improved YOLOv7-tiny, used to implement the detection method according to any one of claims 1 to 9, characterized in that: include: An image acquisition module is used to acquire strawberry images of different growth periods and establish a strawberry image dataset; A data enhancement module, used for performing a data enhancement operation on the strawberry image dataset to obtain a strawberry image enhanced dataset; A model training module is used to input the strawberry image enhancement data set into the improved YOLOv7-tiny strawberry detection model, set a loss function for training, and obtain a trained strawberry detection model; The flower and fruit detection module is used to obtain the strawberry image to be detected, input it into the trained strawberry detection model, and obtain the flower and fruit detection target of the strawberry image to be detected.

Images