Method and device for generating bud detection model

Abstract

The embodiment of the invention provides a bud detection model generation method and device, and the method comprises the steps: replacing a backbone network of a first bud detection model with a deep learning network InceptionNext, and replacing a convolution module in the InceptionNext with a first depth separable convolution module, and obtaining a target backbone network; adding a lightweight visual network CloFormer into the neck network of the first bud detection model, and replacing a convolution module in the CloFormer with a second depth separable convolution module to obtain a target neck network; generating a second bud detection model based on the first bud detection model, the target trunk network and the target neck network; and training the second bud detection model based on the plant sample image set to obtain a target bud detection model, and detecting the growth state of the buds through the target bud detection model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for generating a flower bud detection model. Background Art

[0002] In the field of flower growth monitoring, early flower bud detection methods mainly rely on traditional image processing technology, which usually involves steps such as image acquisition, preprocessing, feature extraction and classification. In a small greenhouse or home garden environment, flower bud detection methods based on traditional image processing can meet the needs to a certain extent, and it is also a relatively low-cost option. However, these methods generally have some limitations, such as low detection accuracy, inability to provide accurate monitoring in the early stages of flower bud growth, and poor adaptability to environmental changes; in addition, these methods cannot provide detailed information such as flower bud growth status, pests and diseases, and often require further manual inspection and intervention, which increases labor costs; and flower bud detection algorithms need to be deployed on handheld embedded hardware devices for easy use, but these devices often have limited computing power and cannot run large network models. Summary of the invention

[0003] In view of the above problems, the embodiments of the present application provide a method and device for generating a flower bud detection model that overcomes the above problems or at least partially solves the above problems.

[0004] In a first aspect, an embodiment of the present application provides a method for generating a flower bud detection model, the method comprising:

[0005] The backbone network of the first flower bud detection model is replaced with the deep learning network InceptionNext, and the convolution module in the InceptionNext is replaced with the first depth-separable convolution module Inception Depth-Wise to obtain the target backbone network;

[0006] Add a lightweight visual network CloFormer to the neck network of the first flower bud detection model, and replace the convolution module in the CloFormer with the second depth-separable convolution module Inception Depth-Wise to obtain a target neck network;

[0007] Generate a second flower bud detection model based on the first flower bud detection model, the target trunk network, and the target neck network;

[0008] The second flower bud detection model is trained based on the plant sample image set to obtain a target flower bud detection model, so as to detect the growth status of the flower bud through the target flower bud detection model.

[0009] Optionally, for the first depth-separable convolution module Inception Depth-Wise and the second depth-separable convolution module Inception Depth-Wise, one-third of the channels use R*R kernel convolution, one-third of the channels use 1*S kernel convolution, and one-third of the channels use S*1 kernel convolution, where S and R are both positive integers.

[0010] Optionally, the CloFormer in the target neck network includes a component Clo block, and the Clo block includes a local branch and a global branch;

[0011] Wherein, the local branch extracts local features of the image through the convolutional attention AttnConv module;

[0012] The global branch extracts low-frequency image global features through an attention module.

[0013] Optionally, the CloFormer in the target neck network further comprises a convolutional feed-forward neural network ConvFFN, wherein the ConvFFN uses a deep convolution after an activation function GELU to aggregate local information;

[0014] The ConvFFN includes a ConvFFN within a stage and a ConvFFN connecting two stages.

[0015] Optionally, the training of the second flower bud detection model based on the plant sample image set to obtain a target flower bud detection model includes:

[0016] Inputting each plant sample image into the target backbone network of the second bud detection model to obtain a first image feature;

[0017] Inputting the first image feature into the target neck network to obtain a second image feature;

[0018] Performing multi-scale feature fusion on the first image feature and the second image feature through the target neck network to obtain a fused feature;

[0019] Inputting the fused features into the head network of the second flower bud detection model to obtain the growth state of the flower bud;

[0020] The second flower bud detection model is trained based on the plant sample image set and the growth state of the flower bud corresponding to each plant sample image to obtain a target flower bud detection model.

[0021] Optionally, the target neck network includes M CloFormers, and the first image feature is input into the target neck network to obtain the second image feature, including:

[0022] Inputting the first image feature into the target neck network, and obtaining M third image features after being processed by the M CloFormers, where M is an integer greater than 1;

[0023] The M third image features are subjected to multi-scale feature fusion through a spatial pyramid structure SPPF to obtain a second image feature.

[0024] In a second aspect, an embodiment of the present application further provides a device for generating a flower bud detection model, the device comprising:

[0025] A first replacement module is used to replace the backbone network of the first flower bud detection model with the deep learning network InceptionNext, and replace the convolution module in the InceptionNext with the first depth-separable convolution module Inception Depth-Wise, so as to obtain a target backbone network;

[0026] A second replacement module is used to add a lightweight visual network CloFormer to the neck network of the first flower bud detection model, and replace the convolution module in the CloFormer with a second depth-separable convolution module InceptionDepth-Wise to obtain a target neck network;

[0027] A generating module, configured to generate a second flower bud detection model based on the first flower bud detection model, the target trunk network and the target neck network;

[0028] A training module is used to train the second flower bud detection model based on a plant sample image set to obtain a target flower bud detection model, so as to detect the growth status of the flower bud through the target flower bud detection model.

[0029] Optionally, for the first depth-separable convolution module Inception Depth-Wise and the second depth-separable convolution module Inception Depth-Wise, one-third of the channels use R*R kernel convolution, one-third of the channels use 1*S kernel convolution, and one-third of the channels use S*1 kernel convolution, where S and R are both positive integers.

[0030] Optionally, the CloFormer in the target neck network includes a component Clo block, and the Clo block includes a local branch and a global branch;

[0031] Wherein, the local branch extracts local features of the image through the convolutional attention AttnConv module;

[0032] The global branch extracts low-frequency image global features through an attention module.

[0033] Optionally, the CloFormer in the target neck network further comprises a convolutional feed-forward neural network ConvFFN, wherein the ConvFFN uses a deep convolution after an activation function GELU to aggregate local information;

[0034] The ConvFFN includes a ConvFFN within a stage and a ConvFFN connecting two stages.

[0035] Optionally, the training module is specifically used to:

[0036] Inputting each plant sample image into the target backbone network of the second bud detection model to obtain a first image feature;

[0037] Inputting the first image feature into the target neck network to obtain a second image feature;

[0038] Performing multi-scale feature fusion on the first image feature and the second image feature through the target neck network to obtain a fused feature;

[0039] Inputting the fused features into the head network of the second flower bud detection model to obtain the growth state of the flower bud;

[0040] The second flower bud detection model is trained based on the plant sample image set and the growth state of the flower bud corresponding to each plant sample image to obtain a target flower bud detection model.

[0041] Optionally, the target neck network includes M CloFormers, and when the training module inputs the first image feature into the target neck network to obtain the second image feature, it is specifically used to:

[0042] Inputting the first image feature into the target neck network, and obtaining M third image features after being processed by the M CloFormers, where M is an integer greater than 1;

[0043] The M third image features are subjected to multi-scale feature fusion through a spatial pyramid structure SPPF to obtain a second image feature.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a transceiver, and a processor:

[0045] A memory for storing a computer program; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer program in the memory and executing the method as described in the first aspect above.

[0046] In a fourth aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method described in the first aspect above.

[0047] In the above embodiment of the present application, the backbone network of the original first flower bud detection model is replaced with a lightweight deep learning network InceptionNext, and the convolution module in the InceptionNext is replaced with the first depth-separable convolution module Inception Depth-Wise to obtain the target backbone network. The backbone network is replaced with a lightweight target backbone network, which simplifies the network structure and computational cost of the flower bud detection model. In addition, a lightweight visual network CloFormer is added to the neck network of the first flower bud detection model, and the convolution module in the CloFormer is replaced with the second depth-separable convolution module Inception Depth-Wise to obtain the target neck network. A context-aware locally enhanced lightweight visual network is added to the neck network, which can focus on capturing high-frequency and low-frequency information, thereby adapting to feature extraction tasks in complex backgrounds and reducing the amount of computation. Furthermore, based on the first bud detection model, the target backbone network and the target neck network, a second bud detection model is generated, and the second bud detection model is trained based on the plant sample image set to obtain a target bud detection model, so that different growth states of the buds can be detected in real time through the target bud detection model, thereby improving the detection performance and efficiency of the bud generation state. The entire solution automatically detects the growth state of the buds, reduces labor costs, and uses a lightweight network structure so that it can be applied to embedded hardware devices, improving the portability of bud detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 A flow chart of a method for generating a flower bud detection model provided in an embodiment of the present application;

[0050] Figure 2 A flow chart for detecting the growth status of a flower bud provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of InceptionNext provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of the target flower bud detection model provided in the embodiment of the present application;

[0053] Figure 5 A schematic diagram of the structure of the Clo block provided in the embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of ConvFFN provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of DWconv provided in an embodiment of the present application;

[0056] Figure 8 A structural block diagram of a device for generating a flower bud detection model provided in an embodiment of the present application;

[0057] Fig. 9 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0059] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0060] The following is a detailed description of the method for generating the flower bud detection model provided in the embodiment of the present application through specific embodiments and their application scenarios in combination with the accompanying drawings.

[0061] Specifically, the present application embodiment provides a method for generating a bud detection model, such as Figure 1 As shown, the following steps may be specifically included:

[0062] Step 101, replacing the backbone network of the first flower bud detection model with the deep learning network InceptionNext, and replacing the convolution module in the InceptionNext with the first depth-separable convolution module Inception Depth-Wise, to obtain a target backbone network.

[0063] Construct a first flower bud detection model, which includes a backbone network Bottleneck, a neck network, a head network, etc. The backbone network is used to extract image features, the neck network is used to fuse multi-scale features, and the head network is used to output flower bud detection results. Among them, the first flower bud detection model can be a YOLOv8 model. The YOLOv8 model is the latest version of the target detection model and belongs to the YOLO (You Only Look Once, target detection algorithm) series. YOLOv8 inherits the advantages of previous generations of algorithms and has made many improvements and innovations, which can provide higher detection speed and accuracy.

[0064] Replacing the original backbone network Bottleneck with the InceptionNext lightweight network can reduce the number of network parameters and make the network lightweight so that it is better not included in embedded hardware devices. InceptionNext proposes to decompose the depth convolution Depth-Wise of the large convolution kernel into four parallel branches along the channel dimension, namely a small rectangular convolution kernel, two orthogonal band convolution kernels and an identity mapping. Specifically, InceptionNext decomposes the large kernel of the deep convolution into small rectangular convolution kernels with the Inception style. Among them, Depth-wise is a special convolution operation, which is characterized by a convolution kernel responsible for only one channel, that is, a channel is convolved by only one convolution kernel, which can reduce the amount of calculation and the number of parameters, thereby improving the calculation speed and efficiency. The channel dimension in Convolutional Neural Networks (CNN) refers to the dimension or depth of the data, specifically, the number of channels of the input data or feature map. Inception style refers to a network structure design style widely used in the field of deep learning.

[0065] The convolution module in the above InceptionNext is replaced with the first depth-separable convolution module InceptionDepth-Wise, that is, the first depth-separable convolution module Inception Depth-Wise is used to replace the traditional convolution module, thereby obtaining a target backbone network, which adopts the InceptionNext network including the first depth-separable convolution module.

[0066] Step 102: Add a lightweight visual network CloFormer to the neck network of the first flower bud detection model, and replace the convolution module in the CloFormer with a second depth-separable convolution module InceptionDepth-Wise to obtain a target neck network.

[0067] A lightweight visual deep learning model Transformer structure (i.e., CloFormer) with context-aware local enhancement is added to the neck network, and the convolution module in the CloFormer is replaced with the second depth-wise separable convolution module Inception Depth-Wise, that is, the second depth-wise separable convolution module Inception Depth-Wise is used to replace the traditional convolution module, and the input and output resolution of the second depth-wise separable convolution module Inception Depth-Wise is consistent with the input and output resolution of the traditional convolution module, thereby obtaining the target neck network, adopting the CloFormer structure with stronger neck addition capability and no time consumption, and using shared weights and context-aware weights to better handle the relationship between different positions in the image, thereby adapting to feature extraction tasks under complex backgrounds.

[0068] Among them, the CloFormer network is a lightweight Vision Transformer architecture used to process image tasks on mobile devices. Vision Transformer is a deep learning model based on the Transformer architecture, mainly used to process computer vision tasks.

[0069] Step 103: Generate a second flower bud detection model based on the first flower bud detection model, the target backbone network and the target neck network.

[0070] The backbone network in the first flower bud detection model is replaced with the target backbone network, the neck network is updated to the target neck network, and the head network remains unchanged, thereby obtaining an updated second flower bud detection model. The updated second flower bud detection model includes: a target backbone network, a target neck network, a head network, etc.

[0071] Step 104: training the second flower bud detection model based on the plant sample image set to obtain a target flower bud detection model, so as to detect the growth status of the flower buds through the target flower bud detection model.

[0072] The video acquisition unit is used to collect plant videos of different bud growth states, and the frame images of the plant videos are used as plant sample images. Multiple plant sample images constitute a plant sample image set. Using the learning transfer method, YOLOv8_COCO_weights is used as the weight of the pre-trained model (i.e., the second bud detection model), and the plant sample image set is used to fine-tune the weight of the pre-trained model. The random gradient descent method is used to optimize the weight, and the second bud detection model is trained using the anchor-free frame-based technology, and finally the target bud detection model is obtained.

[0073] Among them, the weights of the YOLOv8 image dataset (Common Objects in Context, COCO) usually refer to the model weights trained on the COCO dataset. These weights contain the parameters of the model after training on the COCO dataset and can be used for target detection tasks.

[0074] The COCO dataset is a large image dataset containing more than 330,000 images and more than 250,000 annotated objects. It is widely used for tasks such as object detection, segmentation, and pose estimation. YOLOv8 training on the COCO dataset can help the model learn richer object detection capabilities and is suitable for object detection tasks in various practical scenarios.

[0075] In practical applications, the target bud detection model is applied to embedded hardware devices (such as handheld devices), such as Figure 2 As shown, the embedded hardware device collects plant images through a camera and converts the plant images into a preset size (such as: 640*512) through preprocessing. The target bud detection model obtained by training the second bud detection model based on the plant sample image set detects the growth state of the flower buds in the plant image to obtain a detection result about the growth state of the flower buds. The detection result includes the growth state of the flower buds of each category and the corresponding probability value. If the detection result includes the growth state of the flower bud with a probability value greater than 0.5, it is determined that the detection is successful, and the growth state of the flower bud with a probability value greater than 0.5 is determined as the final growth state of the flower bud, and the final growth state of the flower bud is displayed; if the detection result does not include the growth state of the flower bud with a probability value greater than 0.5, it is determined that the detection has failed, and the plant image is re-collected by the camera for detection.

[0076] Among them, the growth status of flower buds includes but is not limited to: spitting bud (SB), balloon bud (BB), expanding blossom (EB), full blossom (FB), early petal fall (EPF), late petal fall (LPF), and stamen (SN).

[0077] In the above embodiment of the present application, the backbone network of the original first flower bud detection model is replaced with a lightweight deep learning network InceptionNext, and the convolution module in the InceptionNext is replaced with the first depth-separable convolution module Inception Depth-Wise to obtain the target backbone network. The backbone network is replaced with a lightweight target backbone network, which simplifies the network structure and computational cost of the flower bud detection model. In addition, a lightweight visual network CloFormer is added to the neck network of the first flower bud detection model, and the convolution module in the CloFormer is replaced with the second depth-separable convolution module Inception Depth-Wise to obtain the target neck network. A context-aware locally enhanced lightweight visual network is added to the neck network, which can focus on capturing high-frequency and low-frequency information, thereby adapting to feature extraction tasks in complex backgrounds and reducing the amount of computation. Furthermore, based on the first bud detection model, the target backbone network and the target neck network, a second bud detection model is generated, and the second bud detection model is trained based on the plant sample image set to obtain a target bud detection model, so that different growth states of the buds can be detected in real time through the target bud detection model, thereby improving the detection performance and efficiency of the bud generation state. The entire solution automatically detects the growth state of the buds, reduces labor costs, and uses a lightweight network structure so that it can be applied to embedded hardware devices, improving the portability of bud detection.

[0078] As an optional specific embodiment, for the first depth-separable convolution module Inception Depth-Wise and the second depth-separable convolution module Inception Depth-Wise, one-third of the channels use R*R kernel convolution, one-third of the channels use 1*S kernel convolution, and one-third of the channels use S*1 kernel convolution, where S and R are both positive integers.

[0079] For the first depth-separable convolution module Inception Depth-Wise in InceptionNext, 1 / 3 of the channels use R*R kernel convolution (e.g. 3×3 kernel convolution), 1 / 3 of the channels use 1*S kernel convolution, and the remaining 1 / 3 of the channels use S*1 kernel convolution. This new first depth-separable convolution module Inception Depth-Wise not only reduces the amount of computation, but also maintains competitive performance.

[0080] For example: Figure 3 As shown in the figure, for the input image of InceptionNext, it is first divided into four groups along the channel dimension, and then these four groups of features are respectively decomposed into four parallel branches along the channel dimension by four different operators (that is, the depth convolution of the large convolution kernel Depth-Wise is decomposed into four parallel branches along the channel dimension, namely Split): small square kernel (for example, 3x3), horizontal ribbon kernel (for example, 1x11), vertical ribbon kernel (for example, 11x1), and identity mapping (no transformation). The output image can be obtained by splicing and normalizing the above four groups of features.

[0081] As another optional specific embodiment, the CloFormer in the target neck network includes a component Cloblock, and the Clo block includes a local branch and a global branch;

[0082] Wherein, the local branch extracts local features of the image through the convolutional attention AttnConv module;

[0083] The global branch extracts low-frequency image global features through an attention module.

[0084] like Figure 4 As shown, in the target neck network, the convolution module is replaced by the second depth-separable convolution module Inception Depth-Wise's CloFormer including Clo block, as shown Figure 5 As shown, Clo block is one of the core components of CloFormer. Clo block consists of local branches and global branches.

[0085] The local branch is mainly responsible for capturing high-frequency local information. It uses a new type of convolution operation called AttnConv. AttnConv combines the styles of convolution operation and attention mechanism, aggregates local information through shared weights, and uses context-aware weights to enhance local features of the image. AttnConv enhances the processing ability of local features of the image through the combination of shared weights and context-aware weights. This design enables AttnConv to effectively process high-frequency local information and inherit the translation and variability characteristics of convolution. Compared with traditional convolution operations, AttnConv can better handle the relationship between different positions in the image. Among them, AttnConv is a lightweight attention-style convolution operator that aggregates local information using shared weights and enhances local features of the image through carefully designed context-aware weights.

[0086] The global branch adopts a standard attention mechanism to extract low-frequency global information by downsampling K and V and then performing a standard attention process on Q, K, and V. The design of the global branch allows the model to learn feature representations at different scales, which is crucial for understanding the global context of the image. The specific calculation formula is as follows:

[0087] Q,K,V=FC(X in )

[0088] X in is the input of AttnConv, FC is a fully connected network, and after performing linear transformation, it first aggregates local image features using shared weights on V. Then, based on the processed V and Q, K, context-aware local enhancement is performed. This dual-branch structure enables CloFormer to capture both high-frequency and low-frequency information, improving its overall perception ability.

[0089] Among them, Q (Query), K (Key), and V (Value) are the core concepts in the attention mechanism, which are used to implement weighted calculations in sequence models. Q stands for query, which is equivalent to the current input or state in the decoder. K stands for key, which is equivalent to the state of all inputs in the encoder. V stands for value, which represents the information or output vector corresponding to each input in the encoder. In the attention mechanism, Q and K are similarly calculated to obtain weights (attention scores), and then these weights are used to perform weighted summation on V to obtain the context vector.

[0090] As another optional specific embodiment, the CloFormer in the target neck network further includes a convolutional feed-forward neural network ConvFFN, and the ConvFFN uses deep convolution after the activation function GELU to aggregate local information;

[0091] The ConvFFN includes a ConvFFN within a stage and a ConvFFN connecting two stages.

[0092] like Figure 4 As shown in the figure, the convolution module is replaced by the second depth-separable convolution module Inception Depth-Wise's CloFormer also includes a convolutional feed-forward neural network (Convolutional Feed-Forward Network, ConvFFN). ConvFFN uses deep convolution (DWconv) after the activation function GELU, so that the network can aggregate local information and perform downsampling operations internally, thereby achieving a lightweight design and excellent context-awareness.

[0093] like Figure 6 As shown in the figure, ConvFFN is a component that is used in CloFormer to replace the traditional feed-forward network (FFN). The main feature of ConvFFN is the use of depthwise convolution (DWconv) after the GELU activation function, which enables ConvFFN to aggregate local information. Due to the existence of DWconv, ConvFFN can directly perform downsampling operations internally without introducing additional PatchMerge operation modules. CloFormer adopts two types of ConvFFN: ConvFFN within the stage and ConvFFN connecting two stages. The ConvFFN within the stage directly uses skip connections; the ConvFFN connecting two stages is mainly used for downsampling operations. This design enables ConvFFN to effectively process local information and can effectively transfer information between different stages. Among them, the DWconv structure is as follows Figure 7 As shown in the figure, the 3-channel input image (3channel input) is processed by three filters to obtain three convolution images Maps.

[0094] The training process of the target flower bud detection model is described below: the step 104 trains the second flower bud detection model based on the plant sample image set to obtain the target flower bud detection model, which specifically includes steps 1041 to 1045:

[0095] Step 1041, input each plant sample image into the target backbone network of the second flower bud detection model to obtain a first image feature.

[0096] Each plant sample image (eg, 640*512*3 RGB image) is input into the target backbone network of the second bud detection model, and the basic features of the plant sample image are extracted through the target backbone network to obtain the first image features.

[0097] Step 1042: input the first image feature into the target neck network to obtain a second image feature.

[0098] like Figure 4 As shown in the figure, the first image feature is input into the target neck network, and further down-sampled and image features are extracted through CloForme. The C2f layer does not perform down-sampling operations, thereby obtaining the second image feature. Among them, the C2f layer is a special convolutional layer used to fuse feature maps of different scales together to improve the accuracy of target detection. Among them, the C2f layer is a CSP Bottleneck structure containing two convolutional layers in YOLOv8, which is used for feature aggregation and model compression. CSP Bottleneck is a network structure used to improve performance in deep learning.

[0099] Step 1043: Perform multi-scale feature fusion on the first image feature and the second image feature through the target neck network to obtain a fused feature.

[0100] The first image feature output by the target backbone network and the second image feature output by the target neck network are connected across layers through the target neck network, and multi-scale feature fusion is further performed to obtain fused features.

[0101] Step 1044, input the fused features into the head network of the second flower bud detection model to obtain the growth status of the flower bud.

[0102] The above fusion features are input into the head network of the second flower bud detection model for detection to obtain the detection result of the flower bud growth state.

[0103] Step 1045: training the second flower bud detection model based on the plant sample image set and the flower bud growth state corresponding to each plant sample image to obtain a target flower bud detection model.

[0104] The video acquisition unit is used to collect plant videos of different bud growth states, and the frame images of the plant videos are used as plant sample images. Multiple plant sample images constitute a plant sample image set. Using the learning transfer method, YOLOv8_COCO_weights is used as the weight of the pre-trained model (i.e., the second bud detection model), and the plant sample image set is used to fine-tune the weight of the pre-trained model. The random gradient descent method is used to optimize the weight. The optimization objective function of the random gradient descent method can be expressed as:

[0105]

[0106] Among them, f(ω) is usually called the loss function, f i (ω) represents the loss function of the i-th plant sample image (the plant sample image set includes n plant sample images, and n is an integer greater than 1); R(ω) is a regular term to avoid overfitting.

[0107] The second flower bud detection model guides the algorithm to adjust the network weights each time it is trained. The formula for gradient update by stochastic gradient descent is as follows:

[0108] ω t =ω t-1 -η t *Δf i (ω t-1 )

[0109] Among them, η t is the learning rate, which guides the algorithm on how to adjust the hyperparameters of the network weights through the gradient of the loss function; Δf i (ω t-1 ) is the global gradient parameter; t refers to the tth training, Δf i (ω t-1 ) is through ω t-1 Got it.

[0110] Each time training, n plant sample images are extracted to form a plant sample image set for distributed computing, and the model parameter update formula can be obtained:

[0111]

[0112] Among them, w is the working node (i.e. the number of training times), η t is the learning rate, ω t is the working node gradient.

[0113] Furthermore, the target neck network in step 1042 includes M CloFormers, and the first image feature is input into the target neck network to obtain the second image feature, specifically including:

[0114] Inputting the first image feature into the target neck network, and obtaining M third image features after being processed by the M CloFormers, where M is an integer greater than 1;

[0115] The M third image features are subjected to multi-scale feature fusion through a spatial pyramid structure SPPF to obtain a second image feature.

[0116] The target neck network includes M CloFormers. After the first image feature is input to each CloFormer, the output feature (i.e., the third image feature) size P i for:

[0117]

[0118] Among them, Input is the resolution of the image input to CloFormer, and i is the i-th CloFormer.

[0119] The final output third image feature is:

[0120] Q i =Conv(P i ),i=3,4,…,6

[0121] Q i is the tensor after the second depth-separable convolution module Inception Depth-Wise of the CloFormer structure, Conv(P i ) indicates that P i Perform 1*1 convolution operation.

[0122] For example: Figure 4 As shown, the target neck network includes 4 CloFormers, corresponding to: Among them, H represents height, W represents width, and N represents the serial number of CloFormer.

[0123] like Figure 4 As shown in the figure, the third image features obtained by each CloFormer are sent to the Spatial Pyramid Pooling-Fast (SPPF) layer for multi-scale feature fusion, thereby achieving accurate detection of objects of different scales. The SPPF layer enhances the adaptability of the model to changes in the size of flower buds by extracting features at multiple scales. In addition, this structure can also effectively integrate feature information from different CloFormers, making the final detection result more accurate and robust.

[0124] Among them, the calculation formula of the third image feature of CloFormer in each layer is as follows:

[0125] W i =Cat(Q i , conv(Q i ))

[0126] Among them, W i represents the spliced ​​image features of the i-th layer CloFormer, Cat represents the splicing operation, conv(Q i) indicates that Q i Perform convolution operation.

[0127]

[0128] Among them, w1 and w2 are the weights of the features to be stacked, ∈ is a very small value, generally 0.0001, to prevent numerical instability; Resize represents the upsampling or downsampling operation for resolution matching. i out Represents the third image feature of the output of the i-th layer CloFormer.

[0129] In summary, the above embodiment of the present application uses the InceptionNext lightweight network to replace the original backbone network Bottleneck, which can reduce the number of network parameters and make the network lightweight so that it does not belong to embedded hardware devices. In addition, the CloFormer structure with stronger neck addition capability and less time-consuming is added to the neck network. The use of shared weights and context-aware weights can better handle the relationship between different positions in the image, thereby adapting to feature extraction tasks in complex backgrounds. The entire solution automatically detects the growth status of flower buds, reduces labor costs, and uses a lightweight network structure so that it can be applied to embedded hardware devices, improving the portability of flower bud detection.

[0130] The above introduces the method for generating the flower bud detection model provided in the embodiment of the present application. The following will introduce the device for generating the flower bud detection model provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0131] like Figure 8 As shown, the embodiment of the present application further provides a device 800 for generating a flower bud detection model, the device comprising:

[0132] A first replacement module 801 is used to replace the backbone network of the first flower bud detection model with the deep learning network InceptionNext, and replace the convolution module in the InceptionNext with the first depth-separable convolution module Inception Depth-Wise, to obtain a target backbone network;

[0133] A second replacement module 802 is used to add a lightweight visual network CloFormer to the neck network of the first flower bud detection model, and replace the convolution module in the CloFormer with a second depth-separable convolution module Inception Depth-Wise to obtain a target neck network;

[0134] A generating module 803, configured to generate a second flower bud detection model based on the first flower bud detection model, the target trunk network and the target neck network;

[0135] The training module 804 is used to train the second flower bud detection model based on the plant sample image set to obtain a target flower bud detection model, so as to detect the growth status of the flower bud through the target flower bud detection model.

[0136] In the above embodiment of the present application, the backbone network of the original first flower bud detection model is replaced with a lightweight deep learning network InceptionNext, and the convolution module in the InceptionNext is replaced with the first depth-separable convolution module Inception Depth-Wise to obtain the target backbone network. The backbone network is replaced with a lightweight target backbone network, which simplifies the network structure and computational cost of the flower bud detection model. In addition, a lightweight visual network CloFormer is added to the neck network of the first flower bud detection model, and the convolution module in the CloFormer is replaced with the second depth-separable convolution module Inception Depth-Wise to obtain the target neck network. A context-aware locally enhanced lightweight visual network is added to the neck network, which can focus on capturing high-frequency and low-frequency information, thereby adapting to feature extraction tasks in complex backgrounds and reducing the amount of computation. Furthermore, based on the first bud detection model, the target backbone network and the target neck network, a second bud detection model is generated, and the second bud detection model is trained based on the plant sample image set to obtain a target bud detection model, so that different growth states of the buds can be detected in real time through the target bud detection model, thereby improving the detection performance and efficiency of the bud generation state. The entire solution automatically detects the growth state of the buds, reduces labor costs, and uses a lightweight network structure so that it can be applied to embedded hardware devices, improving the portability of bud detection.

[0137] Optionally, for the first depth-separable convolution module Inception Depth-Wise and the second depth-separable convolution module Inception Depth-Wise, one-third of the channels use R*R kernel convolution, one-third of the channels use 1*S kernel convolution, and one-third of the channels use S*1 kernel convolution, where S and R are both positive integers.

[0138] Optionally, the CloFormer in the target neck network includes a component Clo block, and the Clo block includes a local branch and a global branch;

[0139] Wherein, the local branch extracts local features of the image through the convolutional attention AttnConv module;

[0140] The global branch extracts low-frequency image global features through an attention module.

[0141] Optionally, the CloFormer in the target neck network further comprises a convolutional feed-forward neural network ConvFFN, wherein the ConvFFN uses a deep convolution after an activation function GELU to aggregate local information;

[0142] The ConvFFN includes a ConvFFN within a stage and a ConvFFN connecting two stages.

[0143] Optionally, the training module 804 is specifically configured to:

[0144] Inputting each plant sample image into the target backbone network of the second bud detection model to obtain a first image feature;

[0145] Inputting the first image feature into the target neck network to obtain a second image feature;

[0146] Performing multi-scale feature fusion on the first image feature and the second image feature through the target neck network to obtain a fused feature;

[0147] Inputting the fused features into the head network of the second flower bud detection model to obtain the growth state of the flower bud;

[0148] The second flower bud detection model is trained based on the plant sample image set and the growth state of the flower bud corresponding to each plant sample image to obtain a target flower bud detection model.

[0149] Optionally, the target neck network includes M CloFormers, and when the training module 804 inputs the first image feature into the target neck network to obtain the second image feature, it is specifically used to:

[0150] Inputting the first image feature into the target neck network, and obtaining M third image features after being processed by the M CloFormers, where M is an integer greater than 1;

[0151] The M third image features are subjected to multi-scale feature fusion through a spatial pyramid structure SPPF to obtain a second image feature.

[0152] In summary, the above embodiment of the present application uses the InceptionNext lightweight network to replace the original backbone network Bottleneck, which can reduce the number of network parameters and make the network lightweight so that it does not belong to embedded hardware devices. In addition, the CloFormer structure with stronger neck addition capability and less time-consuming is added to the neck network. The use of shared weights and context-aware weights can better handle the relationship between different positions in the image, thereby adapting to feature extraction tasks in complex backgrounds. The entire solution automatically detects the growth status of flower buds, reduces labor costs, and uses a lightweight network structure so that it can be applied to embedded hardware devices, improving the portability of flower bud detection.

[0153] It should be noted here that the device for generating the above-mentioned flower bud detection model provided in the embodiment of the present application can implement all the method steps implemented in the embodiment of the method for generating the above-mentioned flower bud detection model, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0154] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0156] like Fig. 9 As shown, an embodiment of the present application further provides an electronic device, including a memory 920, a transceiver 910, and a processor 900:

[0157] Memory 920, used for storing computer programs;

[0158] A transceiver 910, configured to transmit and receive data under the control of the processor;

[0159] The processor 900 is used to read the computer program in the memory and execute the steps of the method for generating a flower bud detection model as described in any of the above embodiments.

[0160] Among them, Fig. 9 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 900 and memory represented by memory 920. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 910 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which may include a wireless channel, a wired channel, an optical cable, and other transmission media. The processor 900 is responsible for managing the bus architecture and general processing, and the memory 920 may store data used by the processor 900 when performing operations.

[0161] The processor 900 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0162] The processor calls the computer program stored in the memory to execute the method for generating the flower bud detection model provided in the embodiment of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0163] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned flower bud detection model generation method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0164] An embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the above-mentioned method for generating a flower bud detection model.

[0165] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0166] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating a flower bud detection model, characterized in that: The method comprises: The backbone network of the first flower bud detection model is replaced with the deep learning network InceptionNext, and the convolution module in the InceptionNext is replaced with the first depth-separable convolution module Inception Depth-Wise to obtain the target backbone network; Add a lightweight visual network CloFormer to the neck network of the first flower bud detection model, and replace the convolution module in the CloFormer with the second depth-separable convolution module Inception Depth-Wise to obtain a target neck network; Generate a second flower bud detection model based on the first flower bud detection model, the target trunk network, and the target neck network; The second flower bud detection model is trained based on the plant sample image set to obtain a target flower bud detection model, so as to detect the growth status of the flower bud through the target flower bud detection model.

2. The method according to claim 1, characterized in that For the first depth-separable convolution module Inception Depth-Wise and the second depth-separable convolution module Inception Depth-Wise, one-third of the channels use R*R kernel convolution, one-third of the channels use 1*S kernel convolution, and one-third of the channels use S*1 kernel convolution, where S and R are both positive integers.

3. The method according to claim 1, characterized in that The CloFormer in the target neck network includes a component Clo block, and the Clo block includes a local branch and a global branch; Wherein, the local branch extracts local features of the image through the convolutional attention AttnConv module; The global branch extracts low-frequency image global features through an attention module.

4. The method according to claim 3, characterized in that The CloFormer in the target neck network also includes a convolutional feed-forward neural network ConvFFN, which uses deep convolution after the activation function GELU to aggregate local information; The ConvFFN includes a ConvFFN within a stage and a ConvFFN connecting two stages.

5. The method according to claim 1, characterized in that The step of training the second flower bud detection model based on the plant sample image set to obtain a target flower bud detection model includes: Inputting each plant sample image into the target backbone network of the second bud detection model to obtain a first image feature; Inputting the first image feature into the target neck network to obtain a second image feature; Performing multi-scale feature fusion on the first image feature and the second image feature through the target neck network to obtain a fused feature; Inputting the fused features into the head network of the second flower bud detection model to obtain the growth state of the flower bud; The second flower bud detection model is trained based on the plant sample image set and the growth state of the flower bud corresponding to each plant sample image to obtain a target flower bud detection model.

6. The method according to claim 5, characterized in that The target neck network includes M CloFormers, and the first image feature is input into the target neck network to obtain the second image feature, including: Inputting the first image feature into the target neck network, and obtaining M third image features after being processed by the M CloFormers, where M is an integer greater than 1; The M third image features are subjected to multi-scale feature fusion through a spatial pyramid structure SPPF to obtain a second image feature.

7. A device for generating a flower bud detection model, characterized in that: The device comprises: A first replacement module is used to replace the backbone network of the first flower bud detection model with the deep learning network InceptionNext, and replace the convolution module in the InceptionNext with the first depth-separable convolution module Inception Depth-Wise, so as to obtain a target backbone network; A second replacement module is used to add a lightweight visual network CloFormer to the neck network of the first flower bud detection model, and replace the convolution module in the CloFormer with a second depth-separable convolution module InceptionDepth-Wise to obtain a target neck network; A generating module, configured to generate a second flower bud detection model based on the first flower bud detection model, the target trunk network and the target neck network; A training module is used to train the second flower bud detection model based on a plant sample image set to obtain a target flower bud detection model, so as to detect the growth status of the flower bud through the target flower bud detection model.

8. The device according to claim 7, characterized in that For the first depth-separable convolution module Inception Depth-Wise and the second depth-separable convolution module Inception Depth-Wise, one-third of the channels use R*R kernel convolution, one-third of the channels use 1*S kernel convolution, and one-third of the channels use S*1 kernel convolution, where S and R are both positive integers.

9. The device according to claim 7, characterized in that The CloFormer in the target neck network includes a component Clo block, and the Clo block includes a local branch and a global branch; Wherein, the local branch extracts local features of the image through the convolutional attention AttnConv module; The global branch extracts low-frequency image global features through an attention module.

10. The device according to claim 7, characterized in that The CloFormer in the target neck network also includes a convolutional feed-forward neural network ConvFFN, which uses deep convolution after the activation function GELU to aggregate local information; The ConvFFN includes a ConvFFN within a stage and a ConvFFN connecting two stages.

11. The device according to claim 7, characterized in that The training module is specifically used for: Inputting each plant sample image into the target backbone network of the second bud detection model to obtain a first image feature; Inputting the first image feature into the target neck network to obtain a second image feature; Performing multi-scale feature fusion on the first image feature and the second image feature through the target neck network to obtain a fused feature; Inputting the fused features into the head network of the second flower bud detection model to obtain the growth state of the flower bud; The second flower bud detection model is trained based on the plant sample image set and the growth state of the flower bud corresponding to each plant sample image to obtain a target flower bud detection model.

12. The device according to claim 11, characterized in that The target neck network includes M CloFormers. When the training module inputs the first image feature into the target neck network to obtain the second image feature, it is specifically used to: Inputting the first image feature into the target neck network, and obtaining M third image features after being processed by the M CloFormers, where M is an integer greater than 1; The M third image features are subjected to multi-scale feature fusion through a spatial pyramid structure SPPF to obtain a second image feature.

13. An electronic device, characterized in that: Including memory, transceiver, processor: Memory for storing computer programs; a transceiver, for transmitting and receiving data under the control of the processor; A processor is used to read the computer program in the memory and execute the method for generating a flower bud detection model as described in any one of claims 1 to 6.

14. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method for generating a flower bud detection model as described in any one of claims 1 to 6.

Images