Strawberry maturity detection method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a strawberry maturity detection method and device, equipment, a storage medium and a program product, and relates to the technical field of computers. By adopting the method, the detection accuracy of the strawberry maturity can be improved. The method comprises the steps of obtaining a sample image set of strawberries, performing image labeling on the sample image set according to the maturity of the strawberries, and obtaining a training set, a verification set and a test set according to the labeled sample image set; training and updating the initial target detection model by using the training set and the verification set to obtain a preliminarily trained target detection model; modifying a neural network architecture of the preliminarily trained target detection model to obtain an improved target detection model; testing and optimizing the improved target detection model by using the test set until an accuracy index of the improved target detection model on the test set meets a threshold condition, and obtaining a trained target detection model; the trained target detection model is used for detecting the maturity of the strawberry to be detected.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting strawberry maturity. Background Art

[0002] Although some progress has been made in strawberry cultivation, the diverse growing environment poses challenges to strawberry picking, including how to accurately identify strawberries of different maturity levels under complex and changing conditions. In addition, due to the high labor costs involved, it is crucial to establish a method to identify and sort strawberries.

[0003] In traditional technologies, common strawberry maturity detection methods include image processing-based methods, which mainly rely on manually designed feature extraction algorithms and classifiers. However, these methods are limited in effectiveness when dealing with complex scenes and changing lighting conditions, and their performance needs to be improved, resulting in low detection accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for detecting strawberry maturity in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting the maturity of strawberries, comprising:

[0006] Obtain a sample image set of strawberries, perform image annotation on the sample image set according to the maturity of the strawberries, and obtain a training set, a validation set, and a test set based on the annotated sample image set;

[0007] Using the training set and the validation set to train and update the initial target detection model to obtain a preliminarily trained target detection model;

[0008] Modifying the neural network architecture of the initially trained target detection model to obtain an improved target detection model;

[0009] The improved target detection model is tested and optimized using the test set until an accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0010] In one embodiment, modifying the neural network architecture of the initially trained object detection model includes:

[0011] The neck stage of the initially trained target detection model is identified, and the upsampling part and the downsampling part of the neck stage are determined; two full-dimensional dynamic convolutions are added to the upsampling part, and the original convolution in the downsampling part is replaced by the full-dimensional dynamic convolution; an improved intersection-over-union ratio is used in the full-dimensional dynamic convolution, and a multi-dimensional attention mechanism in a parallel strategy is utilized to learn the convolution kernel attention across the four-dimensional kernel space on any convolution layer.

[0012] In one embodiment, the training and updating of the initial target detection model using the training set and the validation set to obtain the preliminarily trained target detection model includes:

[0013] The backbone structure of the initial target detection model is identified, and a new target architecture is introduced into the feature extraction network of the backbone structure; feature extraction parameters and a sampling shape image of a convolution kernel are obtained based on the training set and the validation set; based on the feature extraction parameters and the sampling shape image, the feature extraction network is adaptively regularized and progressively learned by means of a variable kernel convolution to obtain the initially trained target detection model.

[0014] In one embodiment, after introducing the new target architecture into the feature extraction network of the backbone structure, the method further includes:

[0015] In the target architecture, the SE (Squeeze-and-Excitation) channel attention mechanism is used to adaptively weight the features of each channel to obtain an adjusted weight coefficient for each channel; the adjusted weight coefficient is activated by a preset activation function to obtain a target weight coefficient, and a channel recalibration operation is performed in the target architecture according to the target weight coefficient.

[0016] In one embodiment, the image annotation of the sample image set according to the maturity of the strawberries includes:

[0017] The maturity state of the strawberry is divided into a mature state and an immature state, and the sample image set is divided into a negative sample image set and a positive sample image set according to the mature state and the immature state; the mature state includes a mature state and a semi-mature state;

[0018] After labeling the sample image set according to the maturity of the strawberries, the method further includes:

[0019] Data cleaning is performed on the negative sample image set until the cleaned negative sample image set and the positive sample image set meet a sample ratio condition.

[0020] In one embodiment, obtaining a training set, a validation set, and a test set based on the labeled sample image set includes:

[0021] The labeled sample image set is split into an initial training set, an initial verification set and the test set according to a preset ratio; the initial training set and the initial verification set are geometrically transformed and color space adjusted to obtain the image-enhanced training set and the verification set.

[0022] In a second aspect, the present application further provides a strawberry maturity detection device, comprising:

[0023] An image annotation module is used to obtain a sample image set of strawberries, annotate the sample image set according to the maturity of the strawberries, and obtain a training set, a validation set, and a test set based on the annotated sample image set;

[0024] A model training module is used to train and update the initial target detection model using the training set and the validation set to obtain a preliminarily trained target detection model;

[0025] an architecture modification module, configured to modify the neural network architecture of the initially trained target detection model to obtain an improved target detection model;

[0026] A testing and optimization module is used to test and optimize the improved target detection model using the test set until the accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0027] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0028] A sample image set of strawberries is obtained, and image annotation is performed on the sample image set according to the maturity of the strawberries. A training set, a validation set, and a test set are obtained based on the annotated sample image set; an initial target detection model is trained and updated using the training set and the validation set to obtain a preliminarily trained target detection model; a neural network architecture of the preliminarily trained target detection model is modified to obtain an improved target detection model; the improved target detection model is tested and optimized using the test set until an accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0030] A sample image set of strawberries is obtained, and image annotation is performed on the sample image set according to the maturity of the strawberries. A training set, a validation set, and a test set are obtained based on the annotated sample image set; an initial target detection model is trained and updated using the training set and the validation set to obtain a preliminarily trained target detection model; a neural network architecture of the preliminarily trained target detection model is modified to obtain an improved target detection model; the improved target detection model is tested and optimized using the test set until an accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0031] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0032] A sample image set of strawberries is obtained, and image annotation is performed on the sample image set according to the maturity of the strawberries. A training set, a validation set, and a test set are obtained based on the annotated sample image set; an initial target detection model is trained and updated using the training set and the validation set to obtain a preliminarily trained target detection model; a neural network architecture of the preliminarily trained target detection model is modified to obtain an improved target detection model; the improved target detection model is tested and optimized using the test set until an accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0033] The above-mentioned strawberry maturity detection method, device, computer equipment, computer-readable storage medium and computer program product are trained and updated by utilizing a training set and a validation set to train and update an initial target detection model to obtain a preliminary trained target detection model; the neural network architecture of the preliminary trained target detection model is then modified to obtain an improved target detection model; the improved target detection model is then tested and optimized using a test set until the accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model, which is used to detect the maturity of strawberries to be tested. The present application improves the initial target detection model so that the improved target detection model combines full-dimensional dynamic evolution, progressive learning strategy and feature enhancement method, can overcome challenges such as obstacle occlusion, illumination changes and shape diversity, and also improves the accuracy and recall rate of the model. By using the improved target detection model, strawberries of different maturity such as ripe, immature and semi-ripe can be accurately identified and distinguished, thereby improving the detection accuracy of strawberry maturity. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 FIG2 is an application environment diagram of a method for detecting strawberry maturity in one embodiment;

[0036] Figure 2 1 is a flow chart of a method for detecting strawberry maturity in one embodiment;

[0037] Figure 3 A schematic diagram of the structure of a trained target detection model in one embodiment;

[0038] Figure 4 A schematic diagram of the intersection-over-union ratio before improvement in one embodiment;

[0039] Figure 5 Schematic diagram of the process of model training and updating steps in one embodiment;

[0040] Figure 6 is a schematic diagram of the structure of a new target architecture in one embodiment;

[0041] Figure 7 Schematic diagram of the structure of the original MBConv module in one embodiment;

[0042] Figure 8 Schematic diagram of the process of SE channel attention mechanism in one embodiment;

[0043] Figure 9 4D schematic diagram of an attention mechanism in one embodiment;

[0044] Figure 10 Schematic diagram of a process for detecting strawberry maturity in a specific embodiment;

[0045] Figure 11 is a structural block diagram of a device for detecting strawberry maturity in one embodiment;

[0046] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0048] Strawberries grow in diverse and spatially dispersed environments, posing challenges for accurate recognition and real-time image processing in complex environments. Currently, the YOLO (You Only Look Once) family of algorithms, a deep learning-based object detection algorithm, is widely popular due to its lightweight parameters and ease of deployment. However, these algorithms require significant computational resources, resulting in performance degradation during deployment, making them unsuitable for low-resource devices. To address this issue, this application proposes a new YOLOv8 object detection model and improves it by replacing the original backbone with the EfficientNetV2 network (an efficient convolutional neural network architecture). To maintain or improve recognition accuracy while maintaining the model's lightweightness, this application also optimizes the Neck structure and introduces the OD-Conv (Object Detection Convolution) architecture. This convolutional neural network architecture uses a parallel strategy to learn attention values within the convolution kernel in four dimensions, enhancing information fusion capabilities. Furthermore, this application introduces Wise-IoU to improve the model's generalization, which facilitates the recognition of small objects in complex environments and improves classification accuracy. Finally, the new model was applied to the strawberry dataset to verify the feasibility of these methods.

[0049] The strawberry maturity detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown in FIG. , the terminal communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be integrated on the server or placed on the cloud or other network servers. Figure 1 In the application environment shown, the terminal can be, but is not limited to, various personal computers, laptops, smart phones, and tablet computers. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0050] In one embodiment, Figure 2 As shown, a method for detecting the maturity of strawberries is provided, and the method is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0051] Step S201: Obtain a sample image set of strawberries, annotate the sample image set according to the maturity of the strawberries, and obtain a training set, a validation set, and a test set based on the annotated sample image set.

[0052] Among them, the maturity of strawberries can include ripe, semi-ripe and unripe.

[0053] Specifically, to ensure the efficiency and accuracy of strawberry ripeness detection, agricultural drone technology can be used for image acquisition. Equipped with a high-definition camera, the drone precisely flies to a designated location, capturing detailed image data of the strawberries and transmitting it to the terminal as a sample image set. After acquisition, the terminal performs rigorous quality screening and preprocessing on the acquired images. During this process, each image is carefully reviewed by a human operator, removing those of poor quality due to blurring, overexposure, or jitter. This meticulous process ensures clear, reliable, and high-quality image material for subsequent damage identification and analysis, effectively improving detection efficiency and accuracy.

[0054] Step S202: train and update the initial target detection model using the training set and the validation set to obtain a preliminarily trained target detection model.

[0055] Specifically, the terminal first performs data augmentation on the training set and validation set, then uses the augmented training set and validation set to train the initial target detection model, and uses the Efficient-NetV2 feature extraction network to replace the original network in the backbone network, thereby reducing the parameter count and making the model lighter.

[0056] Step S203: Modify the neural network architecture of the initially trained target detection model to obtain an improved target detection model.

[0057] Specifically, the terminal modified the neural network architecture of the initially trained object detection model, applying two ODConv (Omni-Scale Dilated Convolution) operations to the upsampling portion of the neck stage and replacing the original Conv (Convolution) structure in the downsampling phase with ODConv, resulting in an improved object detection model. The ODConv model leverages the multi-dimensional attention mechanism in the parallel strategy to provide flexible attention for learning across the four dimensions of the convolution kernel space, improving model performance.

[0058] Step S204: Testing and optimizing the improved target detection model using the test set until the accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of the strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0059] Among them, the trained target detection model is as follows Figure 3As shown in the figure, the model not only has good detection accuracy, but is also more suitable for overlapping and sparse targets; SPPF (Spatial Pyramid Pooling Fusion) in the figure is a technology used to enhance multi-scale information extraction and processing in convolutional neural networks, Upsampling means upsampling, and Concat means splicing or connection.

[0060] Specifically, after training the object detection model, to fully evaluate its performance, the improved model needs to be tested and optimized using a test set until its accuracy on the test set meets the threshold. Accuracy metrics include, but are not limited to, precision, recall, and mean average precision (mAP). Together, these metrics form a comprehensive system for evaluating the model's performance in object detection tasks.

[0061] In addition, in order to ensure robustness when encountering objects and obstacles of different sizes during the recognition process, it is necessary to use the Wise-IoU (Wise-Intersection over Union, an improved weighted intersection over union) with strong robustness. The traditional intersection over union (IoU) represents the intersection and union ratio of two regions, as shown in the following diagram: Figure 4 As shown, where (x gt ,y gt ) represents the coordinates of the upper left corner of the intersection, (x, y) represents the coordinates of the lower right corner of the intersection, W g Indicates the total width, H g Indicates the total length, W i = x - x gt , H i = y gt - y, then the corresponding calculation formula is established as follows:

[0062]

[0063] In the above formula, represents the anchor frame in the traditional intersection-union ratio, (w, h) represents the coordinates of the lower right corner of the intersection, (w gt , h gt ) represents the coordinate of the upper left corner of the intersection. Among them, the anchor frame is B=[xywh] and the target frame is Bgt=[xgtygtwgthgt]. However, IoU has a flaw that when there is no overlap between bounding boxes, the back-propagation gradient disappears and the width of the overlapping area cannot be updated during training.

[0064] Since the calculation of CIoU (Complete Intersection over Union) in the original YOLOv8 model is relatively complex, a lot of computing resources are consumed in the training phase, and the training set contains low-quality examples. If the regression of the bounding box only contains low-quality examples, it will hinder the performance improvement of the entire model. Therefore, this application uses a dynamic non-monotonic focusing mechanism combined with a gradient gain distribution strategy to reduce the harmful gradients of low-quality examples while reducing the competition of high-quality anchors. Since the traditional set metric factor will increase the penalty for low-quality examples and reduce the generalization ability of the model, it is necessary to weaken the penalty of the set metric. Therefore, distance attention is constructed through distance measurement, and WIoU (Weighted Intersection over Union) v1 with a two-layer attention mechanism is obtained. The calculation formula is defined as follows:

[0065]

[0066]

[0067] In the above formula, represents the anchor frame in the weighted intersection-over-union, represents the anchor frame, ∈[1,e], significantly increasing the number b, i.e., the normal quality anchor frame. For the second calculation formula, if the target frame has two anchor frames with the same IoU, the anchor frame with a relatively far center distance Better than the other. Anchor It is also important, so we will pay more attention to the anchor frames with a larger center distance. At the same time, the outlier degree is introduced to define the quality of the anchor frame, and the calculation formula is as follows:

[0068]

[0069] In the above formula, M represents the number of stages. Small outliers mean high-quality anchor frames, so a small gradient gain is required to make the boundary frame regression focus on the anchor frames of ordinary quality. Since the number of ordinary quality anchors is greater than that of high-quality and low-quality anchors, the generalization ability of the model is improved.

[0070] On this basis, is used to construct the coefficient of non-monotonic focusing, so there is a calculation formula as follows:

[0071]

[0072] In the above formula, represents the anchor frame in the modified weighted intersection-over-union, is the sliding average of the momentum m, is the current sliding value of momentum m. Since it is dynamic, the quality partitioning criterion of the anchor frame is also dynamic, so the model makes the gradient gain distribution strategy conform to the current situation.

[0073] In the detection method of the above-mentioned strawberry maturity, the initial target detection model is trained and updated by utilizing a training set and a validation set to obtain a target detection model for preliminary training; the neural network architecture of the target detection model for preliminary training is then modified to obtain an improved target detection model; the improved target detection model is then tested and optimized using a test set until the accuracy index of the improved target detection model on the test set meets a threshold condition, and a trained target detection model is obtained, which is used to detect the maturity of strawberries to be tested. The present application improves the initial target detection model so that the improved target detection model combines full-dimensional dynamic evolution, progressive learning strategy and feature enhancement method, can overcome challenges such as obstacle occlusion, illumination changes and shape diversity, and also improves the accuracy and recall rate of the model. By using the improved target detection model, strawberries of different maturity such as ripe, immature and semi-ripe can be accurately identified and distinguished, thereby improving the detection accuracy of strawberry maturity.

[0074] In one embodiment, Figure 5 As shown, in the above step S202, the initial target detection model is trained and updated using the training set and the validation set to obtain a preliminarily trained target detection model, which specifically includes the following steps:

[0075] Step S501 : Identify the backbone structure of the initial target detection model and introduce a new target architecture into the feature extraction network of the backbone structure.

[0076] Step S502: Obtain feature extraction parameters and a sampling shape image of a convolution kernel according to the training set and the validation set.

[0077] Step S503 , based on the feature extraction parameters and the sampled shape image, adaptive regularization progressive learning is performed on the feature extraction network by means of variable kernel convolution to obtain a preliminarily trained target detection model.

[0078] Among them, the new target architecture can be a Fused-MBConv architecture.

[0079] Among them, the sampling shape image of the convolution kernel can be flexibly adjusted and is no longer limited to a fixed square, and can better adapt to different target shapes.

[0080] It should be noted that the EfficientNetV1 model uses deep convolution in the shallow layers of the network, so when the size of the training image increases significantly, the training speed will slow down. In addition, the depth and width of each model stage are equal, which further leads to slower operation. However, each stage has a different impact on the training speed and number of parameters of the network. The deep convolution of the original EfficientNetV1 model cannot fully utilize the accelerators available in current hardware, resulting in actual application speed lower than theoretical performance. Efficient-NetV2 is used to replace the original network in the backbone network. A new Fused-MBConv architecture (such as Figure 6 ) to improve computational efficiency and leverage existing hardware capabilities, resulting in faster real-world performance.

[0081] Specifically, in the initial training phase, the image size needs to be increased through smaller images and weak regularization, which also makes adding regularization learning more challenging. Therefore, the network enhances progressive learning and introduces adaptive regularization, as shown in Algorithm 1. The training process is divided into M stages. In the first M-1 stages, the model is trained using the image size and regularization amplitude, and then linear interpolation is used to determine the value of each stage. The final M stages use the target image size and regularization, resulting in a preliminarily trained object detection model.

[0082] Algorithm 1: Adaptive regularized progressive learning. The specific steps are as follows:

[0083] 1) Initialize the image size So and regularization φko;

[0084] 2) Final image size Se and regularization φke;

[0085] 3) Total number of training steps N and number of stages M;

[0086] 4) For i=0 To M-1 Do;

[0087] 5) Image size Si = So + {Se - So};

[0088] 6) Regularization Ri = φki = φko+{φke -φko};

[0089] 7) Use Si and Ri to train the model of stage M;

[0090] 8) EndFor.

[0091] In one embodiment, after introducing the new target architecture into the feature extraction network of the backbone structure, the method of the present application further includes the following steps:

[0092] In the target architecture, the SE channel attention mechanism is used to adaptively weight the features of each channel to obtain the adjusted weight coefficient of each channel; the adjusted weight coefficient is activated by a preset activation function to obtain the target weight coefficient, and the channel recalibration operation is performed in the target architecture according to the target weight coefficient.

[0093] Specifically, the terminal will Figure 7 The 1×1 convolution and depthwise convolution of the MBConv (Mobile Inverted BottleneckConvolution) module shown in the figure are combined into a 3×3 convolution. In addition, both structures now include the SE channel attention mechanism, which adjusts the weight value of each channel on the shortcut branch to increase or decrease the importance of different channels during training. The specific process diagram is shown in the figure. Figure 8 shown.

[0094] The SE module uses global average pooling to compress the two-dimensional features of each feature map into a single value. This stage involves a structure with two fully connected layers, where the first layer compresses the channels and reduces their number to reduce the computational load. After the ReLU activation function is activated, the dimensions are restored to the input dimensions at the beginning of the branch after the second fully connected layer. Finally, the weight coefficients are activated by the Sigmoid activation function.

[0095] In one embodiment, in step S203, the neural network architecture of the initially trained object detection model is modified, specifically including the following steps:

[0096] The neck stage of the preliminarily trained object detection model is identified, and the upsampling and downsampling parts of the neck stage are determined; two full-dimensional dynamic convolutions are added to the upsampling part, and the original convolution in the downsampling part is replaced by a full-dimensional dynamic convolution; an improved intersection-over-union ratio is used in the full-dimensional dynamic convolution, and the multi-dimensional attention mechanism in the parallel strategy is used to learn the convolution kernel attention across the four-dimensional kernel space on any convolution layer.

[0097] Among them, the improved intersection-over-union (IoU) can be Wise-IoU with strong robustness.

[0098] It should be noted that traditional convolution utilizes a single static convolution kernel, which is not affected by the input samples. On the other hand, the weight value depends on the input, resulting in input dependency. CondConv is a pioneering work of dynamic convolution, which uses various convolution kernels for different inputs and linearly weights multiple convolution kernels. However, the weight value depends on the input situation, making the dynamic convolution dependent on the input. Based on this model, the current technology improves the attention mechanism in DyConv and uses the SENet method to assign the extracted attention to different convolution kernels. Although previous models only considered the dynamic characteristics of the number of convolution kernels, they ignored the spatial size of each convolution kernel, the number of input channels, and the number of output channels. This limitation leads to low performance of this convolution in large networks. This application adopts full-dimensional dynamic convolution to solve the limitations of previous models. ODConv uses a multi-dimensional attention mechanism and a parallel strategy to learn convolution kernel attention across a four-dimensional kernel space on any convolution layer. The four-dimensional schematic diagram of the attention mechanism is shown in Figure 2. Figure 9 As shown in Figure 2, different attention values are assigned to the convolution parameters of the spatial position, input channel convolution filter, and output channel convolution filter in (a), (b), and (c), respectively. For a single convolution kernel, different values are assigned to n global convolution kernels in (d).

[0099] Specifically, the terminal identifies the neck stage of the pre-trained object detection model and determines the upsampling and downsampling parts of the neck stage. Two ODConv operations are applied to the upsampling part of the neck stage, and the original Conv structure of the downsampling part is replaced by ODConv. The ODConv model utilizes the multi-dimensional attention mechanism in the parallel strategy to provide flexible attention for learning across the four dimensions of the convolution kernel space, which can improve the performance of the object detection model.

[0100] In one embodiment, in the above step S201, image annotation is performed on the sample image set according to the maturity of the strawberries, which specifically includes the following steps:

[0101] The maturity state of strawberries is divided into mature state and immature state. According to the mature state and immature state, the sample image set is divided into negative sample image set and positive sample image set; the mature state includes mature state and semi-mature state;

[0102] After labeling the sample image set according to the maturity of the strawberries, the method of the present application further includes the following steps:

[0103] The negative sample image set is cleaned until the cleaned negative sample image set and the positive sample image set meet the sample ratio condition.

[0104] Specifically, the terminal uses a professional annotation platform to accurately label the strawberry images captured by the drone, clearly classifying the strawberries in the images as "ripe" (positive samples) and "unripe" (negative samples). Once the annotation is complete, the annotated files are integrated with the original image files to construct a complete dataset. Given the limited number of defective samples in real-world scenarios, to ensure sample balance and meet model training requirements, the dataset is first preprocessed to remove redundant negative samples.

[0105] In one embodiment, in the above step S201, obtaining a training set, a validation set, and a test set based on the labeled sample image set specifically includes the following steps:

[0106] The labeled sample image set is split into an initial training set, an initial validation set, and a test set according to a preset ratio; the initial training set and the initial validation set are geometrically transformed and color space adjusted to obtain the image-enhanced training set and validation set.

[0107] Specifically, the terminal can scientifically divide the labeled sample image set into training, validation, and test sets in a ratio of 8:1:1. To further expand the training data, data augmentation techniques such as geometric transformations (such as rotation, flipping, and cropping) and color space adjustments (including changes in hue, saturation, and brightness) are applied to the training and validation sets, effectively improving the generalization ability and accuracy of model training.

[0108] In one embodiment, Figure 10 As shown, a method for detecting the maturity of strawberries in a specific embodiment is provided, which specifically includes the following steps:

[0109] Step S1001: Obtain a sample image set of strawberries, divide the maturity states of the strawberries into a mature state and an unripe state, and divide the sample image set into a negative sample image set and a positive sample image set according to the mature state and the unripe state; perform data cleaning on the negative sample image set until the cleaned negative sample image set and the positive sample image set meet the sample ratio condition.

[0110] Step S1002: Split the labeled sample image set into an initial training set, an initial validation set, and a test set according to a preset ratio; perform geometric transformation and color space adjustment on the initial training set and the initial validation set to obtain an image-enhanced training set and a validation set.

[0111] Step S1003: Identify the backbone structure of the initial target detection model and introduce a new target architecture into the feature extraction network of the backbone structure; use the SE channel attention mechanism in the target architecture to adaptively weight the features of each channel to obtain the adjusted weight coefficient of each channel; activate the adjusted weight coefficient through a preset activation function to obtain the target weight coefficient, and perform channel recalibration in the target architecture according to the target weight coefficient.

[0112] Step S1004: Obtain feature extraction parameters and a sampling shape image of the convolution kernel based on the training set and the validation set; based on the feature extraction parameters and the sampling shape image, perform adaptive regularization progressive learning on the feature extraction network through a variable kernel convolution method to obtain a preliminarily trained target detection model.

[0113] In step S1005, the neck stage of the preliminarily trained target detection model is identified, and the upsampling part and the downsampling part of the neck stage are determined; two full-dimensional dynamic convolutions are added to the upsampling part, and the original convolution in the downsampling part is replaced with the full-dimensional dynamic convolution to obtain an improved target detection model.

[0114] Step S1006, using the test set to test and optimize the improved target detection model until the accuracy index of the improved target detection model on the test set meets the threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of the strawberries to be tested.

[0115] The beneficial effects brought about by the above embodiment are as follows:

[0116] This application proposes an object detection model based on an improved YOLOv8, replacing the traditional backbone structure with the EfficientNetV2 feature extraction network and using ODConv instead of standard convolution. For the loss function, this application implements a dynamic non-monotonic focusing mechanism and introduces Wise-IoU to replace the traditional CIoU. Compared with the original YOLOv8 model, the proposed model shows significant improvements in mAP50 (Mean Average Precision at Intersection over Union 0.5), precision, and recall, by 16.91%, 14.92%, and 8.4%, respectively. The proposed method can more accurately identify strawberries of different ripeness than the original model.

[0117] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0118] Based on the same inventive concept, embodiments of the present application also provide a strawberry maturity detection device for implementing the aforementioned strawberry maturity detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more strawberry maturity detection device embodiments provided below can be found in the limitations of the strawberry maturity detection method described above and will not be further elaborated here.

[0119] In an exemplary embodiment, Figure 11 As shown, a strawberry maturity detection device is provided, comprising:

[0120] An image annotation module 1101 is configured to obtain a sample image set of strawberries, annotate the sample image set according to the maturity of the strawberries, and obtain a training set, a validation set, and a test set based on the annotated sample image set;

[0121] The model training module 1102 is used to train and update the initial target detection model using the training set and the validation set to obtain a preliminarily trained target detection model;

[0122] An architecture modification module 1103 is used to modify the neural network architecture of the initially trained target detection model to obtain an improved target detection model;

[0123] The test optimization module 1104 is used to test and optimize the improved target detection model using the test set until the accuracy index of the improved target detection model on the test set meets the threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of the strawberries to be tested; the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

[0124] In one embodiment, the model training module 1102 is also used to identify the backbone structure of the initial target detection model and introduce a new target architecture into the feature extraction network of the backbone structure; obtain feature extraction parameters and a sampling shape image of the convolution kernel based on the training set and the validation set; based on the feature extraction parameters and the sampling shape image, the feature extraction network is adaptively regularized and progressively learned by means of a variable kernel convolution to obtain a preliminarily trained target detection model.

[0125] In one embodiment, the strawberry maturity detection device also includes a coefficient adjustment module, which is used to use the SE channel attention mechanism in the target architecture to adaptively weight the features of each channel to obtain the adjusted weight coefficient of each channel; activate the adjusted weight coefficient through a preset activation function to obtain the target weight coefficient, and perform channel recalibration operations in the target architecture according to the target weight coefficient.

[0126] In one embodiment, the architecture modification module 1103 is further used to identify the neck stage of the initially trained target detection model and determine the upsampling part and the downsampling part of the neck stage; add two full-dimensional dynamic convolutions in the upsampling part, and replace the original convolution in the downsampling part with the full-dimensional dynamic convolution; use the improved intersection-over-union ratio in the full-dimensional dynamic convolution, and utilize the multi-dimensional attention mechanism in the parallel strategy to learn the convolution kernel attention across the four-dimensional kernel space on any convolution layer.

[0127] In one embodiment, the image annotation module 1101 is further used to divide the maturity state of the strawberry into a mature state and an immature state, and divide the sample image set into a negative sample image set and a positive sample image set according to the mature state and the immature state; the mature state includes a mature state and a semi-mature state; the strawberry maturity detection device also includes a data cleaning module for performing data cleaning on the negative sample image set until the cleaned negative sample image set and the positive sample image set meet the sample ratio condition.

[0128] In one embodiment, the image annotation module 1101 is further used to split the annotated sample image set into an initial training set, an initial validation set, and a test set according to a preset ratio; perform geometric transformation and color space adjustment on the initial training set and the initial validation set to obtain an image-enhanced training set and a validation set.

[0129] Each module in the strawberry ripeness detection device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0130] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 12 As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means. The wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for detecting the ripeness of strawberries. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0131] Those skilled in the art will understand that Figure 12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0132] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0133] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0134] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0135] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0136] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0137] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0138] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting strawberry maturity, characterized in that: The method comprises: Obtain a sample image set of strawberries, perform image annotation on the sample image set according to the maturity of the strawberries, and obtain a training set, a validation set, and a test set based on the annotated sample image set; Using the training set and the validation set to train and update the initial target detection model to obtain a preliminarily trained target detection model; Modifying the neural network architecture of the initially trained target detection model to obtain an improved target detection model; The improved target detection model is tested and optimized using the test set until an accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

2. The method according to claim 1, characterized in that The modifying of the neural network architecture of the initially trained object detection model includes: Identifying a neck stage of the initially trained object detection model and determining an upsampling portion and a downsampling portion of the neck stage; Two full-dimensional dynamic convolutions are added to the upsampling part, and the original convolution in the downsampling part is replaced by the full-dimensional dynamic convolution; the improved intersection-over-union ratio is used in the full-dimensional dynamic convolution, and the multi-dimensional attention mechanism in the parallel strategy is utilized to learn the convolution kernel attention across the four-dimensional kernel space on any convolution layer.

3. The method according to claim 1, characterized in that The initial target detection model is trained and updated using the training set and the validation set to obtain a preliminarily trained target detection model, including: Identifying the backbone structure of the initial object detection model and introducing a new object architecture into the feature extraction network of the backbone structure; Obtaining feature extraction parameters and a sampling shape image of a convolution kernel according to the training set and the validation set; Based on the feature extraction parameters and the sampled shape image, the feature extraction network is adaptively regularized and progressively learned by means of a variable kernel convolution to obtain the preliminarily trained target detection model.

4. The method according to claim 3, characterized in that After introducing the new target architecture into the feature extraction network of the backbone structure, the method further includes: Adaptively weighting the features of each channel using the SE channel attention mechanism in the target architecture to obtain an adjusted weight coefficient for each channel; The adjusted weight coefficient is activated by a preset activation function to obtain a target weight coefficient, and a channel recalibration operation is performed in the target architecture according to the target weight coefficient.

5. The method according to claim 1, wherein The image annotation of the sample image set according to the maturity of the strawberries includes: The maturity state of the strawberry is divided into a mature state and an immature state, and the sample image set is divided into a negative sample image set and a positive sample image set according to the mature state and the immature state; the mature state includes a mature state and a semi-mature state; After labeling the sample image set according to the maturity of the strawberries, the method further includes: Data cleaning is performed on the negative sample image set until the cleaned negative sample image set and the positive sample image set meet a sample ratio condition.

6. The method according to any one of claims 1 to 5, characterized in that The training set, validation set and test set are obtained based on the labeled sample image set, including: Splitting the labeled sample image set into an initial training set, an initial validation set, and the test set according to a preset ratio; Performing geometric transformation and color space adjustment on the initial training set and the initial validation set to obtain the training set and the validation set after image enhancement.

7. A strawberry maturity detection device, characterized in that: The device comprises: An image annotation module is used to obtain a sample image set of strawberries, annotate the sample image set according to the maturity of the strawberries, and obtain a training set, a validation set, and a test set based on the annotated sample image set; A model training module is used to train and update the initial target detection model using the training set and the validation set to obtain a preliminarily trained target detection model; An architecture modification module, configured to modify the neural network architecture of the initially trained target detection model to obtain an improved target detection model; A testing and optimization module is used to test and optimize the improved target detection model using the test set until the accuracy index of the improved target detection model on the test set meets a threshold condition, thereby obtaining a trained target detection model; the trained target detection model is used to detect the maturity of strawberries to be tested; and the maturity detection results of the strawberries to be tested include ripe, unripe, and semi-ripe.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images