Green-husk walnut target detection method, device and equipment and storage medium

Abstract

The invention discloses a green walnut target detection method, device and equipment and a storage medium, and the method comprises the steps: obtaining a green walnut image, and extracting a green walnut pixel image from the green walnut image through a semantic segmentation model; combining the green walnut pixel image with the green walnut image to obtain a green walnut pixel region image; the improved YOLOv8 network model is used for carrying out target detection on the green walnut pixel area image, a detection result is obtained, and the improved YOLOv8 network model is a network model obtained after an SPPCSPC module and a CNeB module are introduced into the YOLOv8 network model. According to the invention, the improved YOLOv8 network model is utilized to carry out target detection on the green walnut pixel region image obtained by combining the green walnut pixel image and the green walnut image, so that compared with the prior art, the green walnut detection effect is effectively improved, and the counting labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a green-skinned walnut target detection method, device, equipment and storage medium. Background Art

[0002] Currently, walnut yield measurement and picking are usually done manually. Since the walnut maturity period is mostly one year, if the yield is measured after the walnuts are picked, it will cause a certain lag in the formulation of many plans and the implementation of policies, which will cause a certain obstacle to the economic scale development of the walnut industry. How to accurately predict the yield of walnuts in advance will play a very important role in the development of the walnut industry.

[0003] With the development of high-tech, visual technology with artificial intelligence as the core has begun to show its high efficiency advantages, such as intelligent statistics of walnut yield in the production process, intelligent identification and monitoring of pests and diseases, and intelligent grading of walnut kernels in the purchase process. The introduction of artificial intelligence has greatly reduced labor costs and working time, improved work efficiency and accuracy, and enhanced the level of intelligent, digital and networked decision-making in the entire walnut production process. The determination of fruit yield of fruit trees can not only assist fruit farmers in understanding the growth of fruits and estimating the overall output value of their production areas, but also provide valuable quantitative basis for planting improvement experiments. At present, there is no very effective method for measuring walnut fruit yield. Most of them rely on traditional manual visual measurement methods for measurement. This method not only has high requirements on the experience and physical strength of the measurement personnel, but also there are factors such as shading and low color contrast when walnuts are on fruit trees, which will cause the measurement personnel to easily miss, re-check, and misdetect in walnut measurement. In addition, if the method of uniform weighing after picking is adopted, it will not only increase labor costs and working time, but also have poor operability, making it difficult to effectively measure the yield of a single walnut tree, and unable to provide accurate and effective data support for the walnut yield increase experiment.

[0004] Therefore, there is an urgent need for a green-skinned walnut target detection method that can effectively improve the detection effect of green-skinned walnuts, thereby avoiding the high labor cost and time-consuming problems caused by manual counting. Summary of the invention

[0005] The main purpose of the present invention is to provide a green-skinned walnut target detection method, device, equipment and storage medium, aiming to solve the technical problems of poor detection effect and high labor cost for green-skinned walnuts in the prior art.

[0006] To achieve the above object, the present invention provides a green-skinned walnut target detection method, the method comprising the following steps:

[0007] Acquire a green-skinned walnut image, and extract a green-skinned walnut pixel image from the green-skinned walnut image using a semantic segmentation model;

[0008] Combining the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel area image;

[0009] The improved YOLOv8 network model is used to perform target detection on the green walnut pixel area image to obtain a detection result. The improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model.

[0010] Optionally, before the step of obtaining the green-skinned walnut image and extracting the green-skinned walnut pixel image from the green-skinned walnut image by using a semantic segmentation model, the step further includes:

[0011] Obtain a green-skinned walnut sample data set, and use a semantic segmentation model to perform pixel extraction on images in the green-skinned walnut sample data set to obtain a green-skinned walnut pixel image data set;

[0012] The YOLOv8 network model is improved to obtain an initial YOLOv8 network model, wherein the initial YOLOv8 network model includes an SPPCSPC module and a CNeB module;

[0013] Performing model training on the initial YOLOv8 network model based on the green walnut pixel image dataset to obtain a training result;

[0014] The model parameters of the initial YOLOv8 network model are optimized according to the training results to obtain an improved YOLOv8 network model.

[0015] Optionally, the SPPCSPC module is an SPP structure used in a YOLOv7 network model, including multiple MaxPool operations in parallel to capture image information at different scales;

[0016] The CNeB module includes a CNeB block and a CNeB connection layer. The CNeB block consists of a group of parallel convolution paths, each of which consists of a series of convolution layers and activation functions. Each convolution path uses different convolution kernels to capture features of different scales to obtain corresponding feature maps. The CNeB connection layer is used to splice feature maps of different convolution paths.

[0017] Optionally, the step of obtaining a green-skinned walnut sample data set, and extracting pixels from images in the green-skinned walnut sample data set using a semantic segmentation model to obtain a green-skinned walnut pixel image data set includes:

[0018] Obtaining a green-skinned walnut sample data set, and performing standardization processing on the green-skinned walnut sample data set to obtain a data set to be processed;

[0019] Performing data enhancement processing on the data set to be processed to obtain a data enhanced data set to be processed;

[0020] The semantic segmentation model is used to extract pixels from the images in the data set to be processed after the data enhancement, so as to obtain a green-skinned walnut pixel image data set.

[0021] Optionally, the step of acquiring the green-skinned walnut image and extracting the green-skinned walnut pixel image from the green-skinned walnut image by using a semantic segmentation model includes:

[0022] Get the green-skinned walnut image;

[0023] Performing standardization processing on the green-skinned walnut image to obtain a green-skinned walnut image to be detected;

[0024] The green-skinned walnut image to be detected is input into the semantic segmentation model to obtain the green-skinned walnut pixel image. The semantic segmentation model is a network model obtained after the hybrid attention mechanism is introduced into the encoder module of the UNet3+ semantic segmentation model.

[0025] Optionally, the step of performing target detection on the green walnut pixel area image using the improved YOLOv8 network model to obtain a detection result includes:

[0026] Inputting the green walnut pixel area image into the improved YOLOv8 network model, performing target detection on the green walnut pixel area image through the target anchor frame, and obtaining a target frame with a label;

[0027] The number of the target frames is counted, and the number of the target frames is used as a detection result.

[0028] Optionally, after the step of optimizing the model parameters of the initial YOLOv8 network model according to the training result to obtain an improved YOLOv8 network model, the step further includes:

[0029] The improved YOLOv8 network model is evaluated by using model evaluation indicators, which include precision, recall, balance score F1 and accuracy. The formula is as follows:

[0030]

[0031] Wherein, the balance score F1 is the harmonic mean of precision and recall, P represents precision, R represents recall, TP represents the number of correctly detected targets, FP represents the number of incorrectly detected targets, FN represents the number of incorrectly missed targets, TN represents the number of actually detected targets, and AP refers to the area under the Precision curve.

[0032] In addition, to achieve the above-mentioned purpose, the present invention also proposes a green-skinned walnut target detection device, the device comprising:

[0033] The first image processing module is used to obtain a green-skinned walnut image and extract a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model;

[0034] A second image processing module is used to combine the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel area image;

[0035] The target detection module is used to perform target detection on the green walnut pixel area image using the improved YOLOv8 network model to obtain a detection result. The improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also proposes a green-skinned walnut target detection device, which includes: a memory, a processor, and a green-skinned walnut target detection program stored in the memory and executable on the processor, and the green-skinned walnut target detection program is configured to implement the steps of the green-skinned walnut target detection method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a green-skinned walnut target detection program is stored. When the green-skinned walnut target detection program is executed by a processor, the steps of the green-skinned walnut target detection method described above are implemented.

[0038] The present invention discloses obtaining a green-skinned walnut image, and extracting a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model; combining the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel region image; performing target detection on the green-skinned walnut pixel region image using an improved YOLOv8 network model to obtain a detection result, wherein the improved YOLOv8 network model is a network model obtained after introducing an SPPCSPC module and a CNeB module into the YOLOv8 network model. Since the present invention extracts the green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model, and then uses the improved YOLOv8 network model to perform target detection on the green-skinned walnut pixel region image obtained by combining the green-skinned walnut pixel image with the green-skinned walnut image, compared with the prior art, the present invention effectively improves the detection effect of green-skinned walnuts, thereby avoiding the problems of high labor cost and time consumption caused by manual counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the first embodiment of the green-skinned walnut target detection method of the present invention;

[0040] Figure 2 It is a schematic diagram of image processing in the green-skinned walnut target detection method of the present invention;

[0041] Figure 3 It is a schematic diagram of the structure of the improved YOLOv8 network model in the green walnut target detection method of the present invention;

[0042] Figure 4 This is a flow chart of a second embodiment of a method for detecting green walnut targets according to the present invention;

[0043] Figure 5 It is a flow chart of the third embodiment of the green-skinned walnut target detection method of the present invention;

[0044] Figure 6 It is a structural block diagram of the first embodiment of the green-skinned walnut target detection device of the present invention;

[0045] Figure 7 It is a structural schematic diagram of a green walnut target detection device in a hardware operating environment involved in an embodiment of the present invention.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] The embodiment of the present invention provides a green walnut target detection method, referring to Figure 1 , Figure 1 It is a flow chart of the first embodiment of the green-skinned walnut target detection method of the present invention.

[0049] In this embodiment, the green walnut target detection method includes steps S10 to S30:

[0050] Step S10: obtaining a green-skinned walnut image, and extracting a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model.

[0051] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication and program running functions used in the green walnut counting scenario, such as a personal computer, a laptop computer, a mobile phone, a smart watch, etc., or an electronic device capable of realizing the above functions, a green walnut target detection device, etc. The green walnut target detection device is taken as an example to illustrate this embodiment and the following embodiments.

[0052] It is understandable that the above-mentioned green-skinned walnut image can be an image collected by a device with a camera function in a natural environment.

[0053] It should be understood that semantic segmentation is an important task in computer vision, and its goal is to assign each pixel in an image to a specific category label so that the image can be segmented into multiple meaningful regions.

[0054] It should be noted that the green-skinned walnut pixel image is an image in which pixels outside the green-skinned walnut fruit are removed using semantic segmentation, and only the green-skinned walnut pixels with clear contours are segmented. The green-skinned walnut pixel image can be an image in which the walnut fruit area is white and the background area is black.

[0055] It needs to be explained that in order to suppress unnecessary regional responses, suppress illumination noise information, and enhance the segmentation effect of the model, the above-mentioned semantic segmentation model can be a network model obtained after the encoder module of the UNet3+ semantic segmentation model introduces a hybrid attention mechanism, and the hybrid attention mechanism includes a channel attention module and a spatial attention module.

[0056] In the specific implementation, step S10 includes: obtaining a green-skinned walnut image; standardizing the green-skinned walnut image to obtain a green-skinned walnut image to be detected; inputting the green-skinned walnut image to be detected into a semantic segmentation model to obtain a green-skinned walnut pixel image, and the semantic segmentation model is a network model obtained after introducing a hybrid attention mechanism into the encoder module of the UNet3+ semantic segmentation model.

[0057] Step S20: combining the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel area image.

[0058] For example, refer to Figure 2 , Figure 2 The figure is a schematic diagram of image processing in the green-skinned walnut target detection method of the present invention. In the figure, from left to right, there are a green-skinned walnut image, a green-skinned walnut pixel image, and a green-skinned walnut pixel area image. The green-skinned walnut image and the green-skinned walnut pixel image are combined to obtain a green-skinned walnut pixel area image.

[0059] Step S30: Use the improved YOLOv8 network model to perform target detection on the green walnut pixel area image to obtain a detection result. The improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model.

[0060] For example, refer to Figure 3 , Figure 3 The structure diagram of the improved YOLOv8 network model in the green walnut target detection method of the present invention is shown in the figure. In the figure, the improved YOLOv8 network model is used to detect the target of the green walnut pixel area image. The improved YOLOv8 network model introduces the SPPCSPC module and the CNeB module on the basis of the YOLOv8 network model.

[0061] It should be noted that the above-mentioned SPPCSPC module is the SPP structure used in YOLOv7, which performs better than SPPF and can adjust the receptive field of the target. When the value of the maximum pooling is larger, the receptive field is larger; when the value of the maximum pooling is smaller, the receptive field is smaller. A large receptive field will obtain some large target features, and a small receptive field will obtain some small target features. By adjusting the value of the maximum pooling to obtain different receptive fields, targets of different sizes in the image can all be extracted with features in the model. After the branches of MaxPool, they are 5, 9, and 13 respectively. Different MaxPools represent that it can handle different objects.

[0062] The CNeB module is designed based on the ConvNeXt structure. It uses an adaptive variable adjustment algorithm to dynamically adjust the size of each convolution kernel in the model. The CNeB module mainly consists of two parts: the CNeB block and the CNeB connection layer. The CNeB block consists of a set of parallel convolution paths, each of which consists of a series of convolution layers and activation functions. Convolution kernels of different sizes and depths are used to capture features of different scales and extract features at different levels of abstraction. After the CNeB block, a CNeB connection layer is introduced, which is used to splice the feature maps of different paths to obtain a more representative feature representation. The spliced ​​feature map is used as the input of the next layer and is further passed to the subsequent network layer.

[0063] It needs to be explained that ConvNeXt is a convolutional neural network model jointly proposed by Facebook AI Research and UC Berkeley. ConvNeXt is a pure convolutional neural network composed of standard convolutional neural network modules. It has the characteristics of high efficiency, high accuracy, strong scalability and very simple design.

[0064] ConvNeXt is an improvement based on ResNet50. It has 4 stages and changes the ratio of the number of blocks in each stage from 3:4:6:3 to 1:1:3:1, the same as Swin Transformer. In addition, in terms of feature map downsampling, ConvNeXt uses the same convolution kernel with a step size of 4 and a size of 4*4 as Swin Transformer. ConvNeXt uses some advanced ideas of the Transformer network to make some adjustments and improvements to the existing classic ResNet50 / 200 network. The convolutional neural network performs better because some of the latest ideas and methods of the Transformer network are incorporated into the current module of the convolutional neural network, combining the best features of the two networks.

[0065] In a specific implementation, the green-skinned walnut pixel area image can be input into an improved YOLOv8 network model, and target detection is performed on the green-skinned walnut pixel area image through a target anchor frame to obtain a target frame with a label; the number of the target frames is counted, and the number of the target frames is used as a detection result.

[0066] It should be understood that the number of the above-mentioned target frames represents the number of green-skinned walnuts in the green-skinned walnut pixel area image.

[0067] The present embodiment discloses obtaining a green-skinned walnut image, and extracting a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model; combining the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel region image; performing target detection on the green-skinned walnut pixel region image using an improved YOLOv8 network model to obtain a detection result, wherein the improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model. Since the present embodiment extracts the green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model, and then uses the improved YOLOv8 network model to perform target detection on the green-skinned walnut pixel region image obtained by combining the green-skinned walnut pixel image with the green-skinned walnut image, compared to the prior art, the present embodiment effectively improves the detection effect of green-skinned walnuts, thereby avoiding the problems of high labor costs and time consumption caused by manual counting.

[0068] refer to Figure 4 , Figure 4 This is a flow chart of the second embodiment of the green-skinned walnut target detection method of the present invention.

[0069] Based on the first embodiment described above, in this embodiment, before step S10, steps S01 to S04 are further included:

[0070] Step S01: obtaining a green-skin walnut sample data set, and using a semantic segmentation model to perform pixel extraction on images in the green-skin walnut sample data set to obtain a green-skin walnut pixel image data set.

[0071] Step S02: Improve the YOLOv8 network model to obtain an initial YOLOv8 network model, wherein the initial YOLOv8 network model includes an SPPCSPC module and a CNeB module.

[0072] Step S03: Performing model training on the initial YOLOv8 network model based on the green walnut pixel image dataset to obtain training results.

[0073] Step S04: Optimize the model parameters of the initial YOLOv8 network model according to the training results to obtain an improved YOLOv8 network model.

[0074] It should be noted that the above-mentioned green-skinned walnut sample data set can be a green-skinned walnut image data set obtained from a third-party website, and the green-skinned walnut sample data set can include image data of five categories, such as different degrees of occlusion of branches and leaves, different degrees of fruit overlap, different light intensities, different fruit sizes, and different fruit quantities. Among them, different degrees of leaf occlusion refer to the area of ​​the fruit area of ​​the green-skinned walnut fruit blocked by leaves is less than or greater than 75%; different fruit overlap refers to the area of ​​the fruit that is mutually blocked by the green-skinned walnut fruit is less than or greater than 75%; different light intensities refer to different illuminations of sunlight on the green-skinned walnut, and the shooting scenes are sunny and cloudy, and the shooting times are 9 o'clock, 14 o'clock, and 17 o'clock; different fruit sizes refer to the ratio of the length and width of the fruit in the image to the length and width of the image is less than or greater than 25%; different fruit quantities refer to various numbers of fruits in the image, taking (1-6), (6-12), and (more than 12).

[0075] In order to determine the performance of the improved YOLOv8 network model, after step S04, the method further includes: using a model evaluation index to evaluate the improved YOLOv8 network model. The model evaluation index includes: precision, recall, balance score F1, and accuracy. The formula is as follows:

[0076]

[0077] Wherein, the balance score F1 is the harmonic mean of precision and recall, P represents precision, R represents recall, TP represents the number of correctly detected targets, FP represents the number of incorrectly detected targets, FN represents the number of incorrectly missed targets, TN represents the number of actually detected targets, and AP refers to the area under the Precision curve.

[0078] This embodiment discloses obtaining a green-skinned walnut sample data set, and using a semantic segmentation model to extract pixels from images in the green-skinned walnut sample data set to obtain a green-skinned walnut pixel image data set; improving a YOLOv8 network model to obtain an initial YOLOv8 network model, wherein the initial YOLOv8 network model includes an SPPCSPC module and a CNeB module; performing model training on the initial YOLOv8 network model based on the green-skinned walnut pixel image data set to obtain a training result; optimizing model parameters of the initial YOLOv8 network model according to the training result to obtain an improved YOLOv8 network model; and using a model evaluation index to perform model evaluation on the improved YOLOv8 network model. This embodiment realizes obtaining an improved YOLOv8 network model based on the YOLOv8 network model, and evaluating the improved YOLOv8 network model. Compared with the prior art, this embodiment ensures the reliability of green-skinned walnut target detection.

[0079] refer to Figure 5 , Figure 5 It is a flow chart of the third embodiment of the green-skinned walnut target detection method of the present invention.

[0080] Based on the above embodiments, in this embodiment, step S01 includes steps S011 to S013:

[0081] Step S011: obtaining a green-skinned walnut sample data set, and performing standardization processing on the green-skinned walnut sample data set to obtain a data set to be processed.

[0082] Step S012: performing data enhancement processing on the data set to be processed to obtain a data enhanced data set to be processed.

[0083] Step S013: using a semantic segmentation model to extract pixels from the images in the data set to be processed after data enhancement, to obtain a green-skinned walnut pixel image data set.

[0084] It should be noted that in order to provide sufficient sample data for subsequent model training and thus improve the generalization ability of the model, the original images of green walnuts in the processing data set can be enhanced by methods such as image rotation, left-right swapping, image zooming in and out, small block deformation, random brightness enhancement / reduction, random color / contrast enhancement / reduction, random shearing, random cropping, random flipping, random grayscale coefficient, random fogging, and Gaussian noise.

[0085] This embodiment discloses obtaining a green-skinned walnut sample data set, performing standardization processing on the green-skinned walnut sample data set to obtain a data set to be processed; performing data enhancement processing on the data set to be processed to obtain a data-enhanced data set to be processed; and performing pixel extraction on images in the data-enhanced data set to be processed using a semantic segmentation model to obtain a green-skinned walnut pixel image data set. Since this embodiment performs data enhancement processing on the data set to be processed to obtain a data-enhanced data set to be processed, and then performs pixel extraction on images in the data-enhanced data set to be processed using a semantic segmentation model to obtain a green-skinned walnut pixel image data set, compared to the prior art, this embodiment provides sufficient sample data for subsequent model training, thereby improving the generalization ability of the model.

[0086] In addition, an embodiment of the present invention further proposes a storage medium, on which a green-skinned walnut target detection program is stored. When the green-skinned walnut target detection program is executed by a processor, the steps of the green-skinned walnut target detection method described above are implemented.

[0087] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the green-skinned walnut target detection device of the present invention.

[0088] like Figure 6 As shown, the green-skinned walnut target detection device proposed in the embodiment of the present invention includes: a first image processing module 601, a second image processing module 602 and a target detection module 603.

[0089] The first image processing module 601 is used to obtain a green-skinned walnut image and extract a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model.

[0090] The second image processing module 602 is used to combine the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel area image.

[0091] The target detection module 603 is used to perform target detection on the green walnut pixel area image using the improved YOLOv8 network model to obtain a detection result. The improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model.

[0092] The first image processing module 601 is also used to obtain a green-skinned walnut image; standardize the green-skinned walnut image to obtain a green-skinned walnut image to be detected; input the green-skinned walnut image to be detected into a semantic segmentation model to obtain a green-skinned walnut pixel image, and the semantic segmentation model is a network model obtained after introducing a hybrid attention mechanism into the encoder module of the UNet3+ semantic segmentation model.

[0093] The target detection module 603 is also used to input the green walnut pixel area image into the improved YOLOv8 network model, perform target detection on the green walnut pixel area image through the target anchor frame, and obtain a target frame with a label; count the number of the target frames, and use the number of the target frames as the detection result.

[0094] The present device embodiment discloses obtaining a green-skinned walnut image, and extracting a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model; combining the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel region image; performing target detection on the green-skinned walnut pixel region image using an improved YOLOv8 network model to obtain a detection result, wherein the improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model. Since the present device embodiment extracts the green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model, and then uses the improved YOLOv8 network model to perform target detection on the green-skinned walnut pixel region image obtained by combining the green-skinned walnut pixel image with the green-skinned walnut image, compared to the prior art, the present device embodiment effectively improves the detection effect of green-skinned walnuts, thereby avoiding the problems of high labor costs and time consumption caused by manual counting.

[0095] Based on the above-mentioned first embodiment of the green-skinned walnut target detection device of the present invention, a second embodiment of the green-skinned walnut target detection device of the present invention is proposed.

[0096] In this embodiment, the first image processing module 601 is also used to obtain a green-skinned walnut sample data set, and use a semantic segmentation model to extract pixels from images in the green-skinned walnut sample data set to obtain a green-skinned walnut pixel image data set; improve the YOLOv8 network model to obtain an initial YOLOv8 network model, and the initial YOLOv8 network model includes an SPPCSPC module and a CNeB module; perform model training on the initial YOLOv8 network model based on the green-skinned walnut pixel image data set to obtain a training result; optimize the model parameters of the initial YOLOv8 network model according to the training result to obtain an improved YOLOv8 network model.

[0097] The first image processing module 601 is also used to perform model evaluation on the improved YOLOv8 network model using model evaluation indicators, and the model evaluation indicators include: precision, recall, balance score F1 and accuracy.

[0098] Other embodiments or specific implementations of the green-skinned walnut target detection device of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0099] The present application provides a green-skinned walnut target detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the green-skinned walnut target detection method in the above-mentioned embodiment one.

[0100] Reference below Figure 7 , which shows a schematic diagram of the structure of a green walnut target detection device suitable for implementing the embodiment of the present application. The green walnut target detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The green walnut target detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0101] like Figure 7As shown, the green walnut target detection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the green walnut target detection device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the green walnut target detection device to communicate wirelessly or wired with other devices to exchange data. Although the green walnut target detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0102] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0103] The green-skinned walnut target detection device provided by the present application adopts the green-skinned walnut target detection method in the above embodiment, which can solve the technical problems of poor detection effect and high labor cost for green-skinned walnuts in the prior art. Compared with the prior art, the beneficial effects of the green-skinned walnut target detection device provided by the present application are the same as the beneficial effects of the green-skinned walnut target detection method provided by the above embodiment, and other technical features in the green-skinned walnut target detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0104] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0105] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0106] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0107] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0109] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A green walnut target detection method, characterized in that: The method includes: Acquire a green-skinned walnut image, and extract a green-skinned walnut pixel image from the green-skinned walnut image using a semantic segmentation model; Combining the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel area image; The improved YOLOv8 network model is used to perform target detection on the green walnut pixel area image to obtain a detection result. The improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model.

2. The method according to claim 1, characterized in that Before the step of obtaining the green-skinned walnut image and extracting the green-skinned walnut pixel image from the green-skinned walnut image by using a semantic segmentation model, the method further includes: Obtain a green-skinned walnut sample data set, and use a semantic segmentation model to perform pixel extraction on images in the green-skinned walnut sample data set to obtain a green-skinned walnut pixel image data set; The YOLOv8 network model is improved to obtain an initial YOLOv8 network model, wherein the initial YOLOv8 network model includes an SPPCSPC module and a CNeB module; Performing model training on the initial YOLOv8 network model based on the green walnut pixel image dataset to obtain a training result; The model parameters of the initial YOLOv8 network model are optimized according to the training results to obtain an improved YOLOv8 network model.

3. The method according to any one of claims 1 to 2, characterized in that: The SPP module is the SPP structure used in the YOLOv7 network model, including multiple MaxPool operations in parallel to capture image information from different scales; The CNeB module includes a CNeB block and a CNeB connection layer. The CNeB block consists of a group of parallel convolution paths, each of which consists of a series of convolution layers and activation functions. Each convolution path uses different convolution kernels to capture features of different scales to obtain corresponding feature maps. The CNeB connection layer is used to splice feature maps of different convolution paths.

4. The method according to claim 2, characterized in that The step of obtaining a green-skinned walnut sample data set, and extracting pixels from images in the green-skinned walnut sample data set using a semantic segmentation model to obtain a green-skinned walnut pixel image data set includes: Obtaining a green-skinned walnut sample data set, and performing standardization processing on the green-skinned walnut sample data set to obtain a data set to be processed; Performing data enhancement processing on the data set to be processed to obtain a data enhanced data set to be processed; The semantic segmentation model is used to extract pixels from the images in the data set to be processed after the data enhancement to obtain a green-skinned walnut pixel image data set.

5. The method according to claim 1, characterized in that The step of obtaining a green-skinned walnut image and extracting a green-skinned walnut pixel image from the green-skinned walnut image by using a semantic segmentation model comprises: Get the green-skinned walnut image; Performing standardization processing on the green-skinned walnut image to obtain a green-skinned walnut image to be detected; The green-skinned walnut image to be detected is input into the semantic segmentation model to obtain the green-skinned walnut pixel image. The semantic segmentation model is a network model obtained after the hybrid attention mechanism is introduced into the encoder module of the UNet3+ semantic segmentation model.

6. The method according to claim 1, characterized in that The step of using the improved YOLOv8 network model to perform target detection on the green walnut pixel area image to obtain the detection result includes: Inputting the green walnut pixel area image into the improved YOLOv8 network model, performing target detection on the green walnut pixel area image through the target anchor frame, and obtaining a target frame with a label; The number of the target frames is counted, and the number of the target frames is used as a detection result.

7. The method according to claim 2, characterized in that After the step of optimizing the model parameters of the initial YOLOv8 network model according to the training results to obtain an improved YOLOv8 network model, the method further includes: The improved YOLOv8 network model is evaluated by using model evaluation indicators, which include precision, recall, balance score F1 and accuracy. The formula is as follows: Wherein, the balance score F1 is the harmonic mean of precision and recall, P represents precision, R represents recall, TP represents the number of correctly detected targets, FP represents the number of incorrectly detected targets, FN represents the number of incorrectly missed targets, TN represents the number of actually detected targets, and AP refers to the area under the Precision curve.

8. A green walnut target detection device, characterized in that: The device comprises: The first image processing module is used to obtain a green-skinned walnut image and extract a green-skinned walnut pixel image from the green-skinned walnut image through a semantic segmentation model; A second image processing module is used to combine the green-skinned walnut pixel image with the green-skinned walnut image to obtain a green-skinned walnut pixel area image; The target detection module is used to perform target detection on the green walnut pixel area image using the improved YOLOv8 network model to obtain a detection result. The improved YOLOv8 network model is a network model obtained after introducing the SPPCSPC module and the CNeB module into the YOLOv8 network model.

9. A green walnut target detection device, characterized in that: The device includes: a memory, a processor, and a green-skinned walnut target detection program stored in the memory and executable on the processor, wherein the green-skinned walnut target detection program is configured to implement the steps of the green-skinned walnut target detection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a green-skinned walnut target detection program, and when the green-skinned walnut target detection program is executed by the processor, the steps of the green-skinned walnut target detection method as described in any one of claims 1 to 7 are implemented.

Images