Method, device, equipment and medium for locating key parts of weeds

Abstract

The present invention relates to the technical field of computer image processing, and in particular to a method, device, equipment and medium for locating key parts of weeds. A first weed image data set is obtained; weed and crop images are annotated and data enhanced to obtain a second weed image data set; an improved yolov8-pose network model is built, and the second weed image data set is used for training to obtain a trained yolov8-pose network model; an image of weeds to be detected is obtained, and the trained yolov8-pose network model is input to obtain the pixel coordinates of the top meristem of the weeds; wherein, the improvement of the network model includes: replacing the Bottleneck layer in the C2f convolution block with the MBConV layer to obtain the C2f_EMBC layer; and replacing the loss function with MPDIoU. In this way, the key parts of the top meristem of the weeds are accurately located.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a method, device, equipment and medium for locating key parts of weeds. Background Art

[0002] Weeds, as a major hazard affecting agricultural production, rob crops of sunlight, nutrients, water and space, causing a waste of resources and leading to reduced crop yields. Although herbicides can quickly and easily inhibit the growth of weeds, excessive use can cause ecological damage, and pesticide residues can also affect human health.

[0003] In recent years, various pesticide-free weed control technologies have emerged, and laser technology has gradually been applied to various fields. Based on this, a new weed control method that combines laser technology with visual technology has emerged. This weed control method places higher requirements on visual technology, which not only requires identifying weeds in visual images, but also requires accurate positioning of key parts such as the apical meristem of the weeds.

[0004] Traditional weed recognition technology relies on plant features such as color, shape, and texture for recognition, but such methods have complex algorithms, poor recognition results, and poor generalization capabilities. In recent years, deep learning technology has developed rapidly, and using deep learning to identify weeds has become a trend and has broad application prospects. Common network models include the YOLO series of algorithms, Faster R-CNN, and SSD, which are widely used due to their fast detection speed and high recognition accuracy. At present, research on field weed detection, whether it is traditional visual methods or deep learning-based methods, focuses on identifying the area where the weeds are located, which is not applicable to laser weeding. Summary of the invention

[0005] In order to solve the problem of how to locate key parts such as the apical meristem of weeds, the present invention provides a method, device, equipment and medium for locating key parts of weeds.

[0006] In a first aspect, the present invention provides a method for locating key parts of weeds, comprising the following steps:

[0007] Acquire a first weed image dataset;

[0008] Annotating and data enhancing the weed and crop images in the first weed image dataset to obtain a second weed image dataset;

[0009] Building an improved yolov8-pose network model, and training the improved yolov8-pose network model using the second weed image dataset to obtain a trained yolov8-pose network model;

[0010] Acquire an image of weeds to be inspected;

[0011] Inputting the image of the weeds to be detected into the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the weeds;

[0012] Among them, the improved parts of the improved yolov8-pose network model include:

[0013] Replace the Bottleneck layer in the C2f convolutional block with the MBConV layer to obtain the C2f_EMBC layer;

[0014] Replace the loss function with MPDIoU;

[0015] The improved yolov8-pose network model is obtained.

[0016] In some embodiments, the labeling and data enhancement of the weed and crop images in the first weed image dataset includes:

[0017] The weed and crop images are processed by changing the image hue, changing the image brightness, rotating the images, and performing mosaic occlusion processing to obtain a second weed image data set after enhancement processing.

[0018] In some embodiments, the MBConV layer adopts an inverted residual structure to improve the representation ability of the model and reduce the computational cost, wherein:

[0019] The computational cost of the depth-separable convolution kernel of the MBConV layer is:

[0020] ;

[0021] Among them, h is the feature map height, w is the feature map width, C out is the number of output channels, k is the convolution kernel size, and n is the multiplication factor.

[0022] In some embodiments, the calculation formula of the MPDIoU is:

[0023] ;

[0024] ;

[0025] ;

[0026] in, is the x coordinate of the upper left point of the real target box, To predict the x coordinate of the upper left point of the target box, is the y coordinate of the upper left point of the true value target box, is the y coordinate of the upper left point of the predicted value target box, is the x coordinate of the lower right point of the real target box, To predict the x coordinate of the lower right point of the target box, is the y coordinate of the lower right point of the true value target box, is the y coordinate of the lower right point of the predicted target box, d1 is the distance between the upper left point of the real target box and the predicted target box, d2 is the distance between the lower right point of the real target box and the predicted target box, w is the width of the real target box, and h is the height of the real target box. It is the intersection-over-union ratio of the true target box and the predicted target box.

[0027] In some embodiments, the improved yolov8-pose network model includes: a backbone network, a neck network and a head network; wherein,

[0028] The backbone network includes a first Conv layer, a second Conv layer, a first C2f_EMBC layer, a third Conv layer, a second C2f_EMBC layer, a fourth Conv layer, a third C2f_EMBC layer, a fifth Conv layer, a fourth C2f_EMBC layer and a SPFF layer connected in sequence;

[0029] The neck network includes a first Upsample layer, a first Concat layer, a fifth C2f_EMBC layer, a second Upsample layer, a second Concat layer, a sixth C2f_EMBC layer, a sixth Conv layer, a third Concat layer, a seventh C2f_EMBC layer, a seventh Conv layer, a fourth Concat layer and an eighth C2f_EMBC layer connected in sequence, the first Concat layer can receive the output of the third C2f_EMBC layer for splicing, and the second Concat layer can receive the output of the second C2f_EMBC layer for splicing;

[0030] The head network includes: a first Detect layer, a first Pose layer, a second Detect layer, a second Pose layer, a third Detect layer and a third Pose layer, the first Detect layer and the first Pose layer are connected to the sixth C2f_EMBC layer, the second Detect layer and the second Pose layer are connected to the seventh C2f_EMBC layer, the third Detect layer and the third Pose layer are connected to the eighth C2f_EMBC layer, and the third Concat layer can receive the output of the fifth C2f_EMBC layer for splicing.

[0031] In some embodiments, replacing the Bottleneck layer in the C2f convolution block with the MBConV layer to obtain the C2f_EMBC layer further includes: replacing the SE layer in the MBConV layer with the EfficientSE layer;

[0032] The calculation formula of the EfficientSE layer is:

[0033] ;

[0034] Among them, Nc is the attention function, F is the feature map, σ is the activation function, W0 is the fully connected network, is average pooling, is the maximum pooling.

[0035] In some embodiments, the network structure of the C2f_EMBC layer includes: a ninth Conv layer, a Split layer, at least two MBConv layers, a fifth Concat layer, and a tenth Conv layer connected in sequence; wherein,

[0036] The network structure of the MBConv layer includes: a first convolutional layer, a depth convolutional layer, an eleventh Conv layer, an efficientSE layer, a second convolutional layer and a Dropout layer connected in sequence.

[0037] In a second aspect, the present invention provides a weed key position positioning device, comprising:

[0038] Image acquisition module: used to acquire a first weed image data set and acquire an image of weeds to be detected;

[0039] A data enhancement module: used for labeling and data enhancement of the weed and crop images in the first weed image dataset to obtain a second weed image dataset;

[0040] Model training module: used for building an improved yolov8-pose network model, and training the improved yolov8-pose network model with the second weed image data set to obtain a trained yolov8-pose network model;

[0041] Target detection module: used to detect the weed image to be detected through the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the weed.

[0042] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a program that can be run on the processor, and when the program is executed by the processor, the method for locating key parts of weeds described in the first aspect is implemented.

[0043] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, and when the computer program is executed by at least one processor, the method for locating key parts of weeds described in the first aspect is implemented.

[0044] To solve the existing problems, the present invention has the following advantages:

[0045] Through the improved yolov8-pose network model, not only can we identify weeds, but also locate the key parts of weeds. The positioning of the key parts of weeds, namely the apical meristem, can be applicable to weeds of various forms, so that laser weeding has excellent effect when combined with laser technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a method for locating key weed parts is shown;

[0047] Figure 2 A module block diagram of a weed key part detection device is shown;

[0048] Figure 3 The network structure diagram of the improved yolov8-pose network model is shown;

[0049] Figure 4 The network structure diagram of the C2f_EMBC layer and the MBConv layer is shown;

[0050] Figure 5 The effect diagram of using the improved yolov8-pose network model for detection is shown;

[0051] Figure 6 A comparison chart of detecting irregular weeds using the method for locating the key weed parts based on the improved yolov8-pose and the method for using the contour center as the key weed part is shown;

[0052] Figure 7 A comparison chart of the weed key part positioning method based on improved yolov8-pose and the irregular weed detection method using the contour center as the weed key part under the interference of ambient light changes is shown. DETAILED DESCRIPTION

[0053] The present disclosure will now be discussed with reference to several exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and thus implement the present disclosure, rather than implying any limitation on the scope of the present disclosure.

[0054] As used herein, the term "including" and its variants are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "based at least in part on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment". The term "another embodiment" is to be interpreted as "at least one other embodiment". The orientation or position relationship indicated by the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inner", "outer", "vertical", "horizontal", "lateral", "longitudinal" and the like is based on the orientation or position relationship shown in the accompanying drawings. These terms are mainly for better describing the present application and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific orientation, or to be constructed and operated in a specific orientation. In addition, in addition to being used to indicate an orientation or position relationship, some of the above terms may also be used to indicate other meanings, such as the term "upper" may also be used to indicate a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to the specific circumstances. In addition, the terms "install", "set", "provided with", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection, or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be an internal connection between two devices, elements or components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances. In addition, the terms "first", "second", etc. are mainly used to distinguish different devices, elements or components (the specific types and structures may be the same or different), and are not used to indicate or imply the relative importance and quantity of the indicated devices, elements or components. Unless otherwise specified, "plurality" means two or more.

[0055] In the first aspect, this embodiment discloses a method for locating key parts of weeds, such as Figure 1 As shown, the following steps are included:

[0056] Acquire a first weed image dataset;

[0057] Annotating and data enhancing the weed and crop images in the first weed image dataset to obtain a second weed image dataset;

[0058] Building an improved yolov8-pose network model, and training the improved yolov8-pose network model using the second weed image dataset to obtain a trained yolov8-pose network model;

[0059] Acquire an image of weeds to be inspected;

[0060] Inputting the image of the weeds to be detected into the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the weeds;

[0061] Among them, the improved parts of the improved yolov8-pose network model include:

[0062] Replace the Bottleneck layer in the C2f convolutional block with the MBConV layer to obtain the C2f_EMBC layer;

[0063] Replace the loss function with MPDIoU;

[0064] The improved yolov8-pose network model is obtained.

[0065] In this embodiment, if Figure 1 As shown in the figure, a method for locating the key parts of weeds is proposed. First, the first weed image dataset is acquired by photographic and video equipment such as industrial cameras, and the images of weeds and crops are annotated and data enhanced to obtain the second weed image dataset. Secondly, an improved yolov8-pose network model is built and trained using the second weed image dataset to obtain a trained yolov8-pose network model. Finally, the weed image to be detected is detected by the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the grass.

[0066] Specifically, we collected image data of crops and weeds in the field through photographic and video equipment such as industrial cameras, converted the collected videos into images frame by frame, and randomly selected a batch of images as the first weed image dataset. Then, we used labelme software to frame the apical meristems of crops and weeds in the images, and enhanced the images to obtain the second weed image dataset.

[0067] Specifically, yolov8 (You Only Look Once version 8) is a target detection model that supports multiple visual AI tasks such as image classification, object detection, instance segmentation, pose estimation and tracking. Based on the yolov8-pose network model, the present invention replaces the Bottleneck layer in the C2f convolution block with the MBConV layer, named C2f_EMBC layer, and replaces the loss function in yolov8 with MPDIoU to obtain an improved yolov8-pose network model.

[0068] Specifically, MBConv (Mobile Inverted Bottleneck Convolution) is a convolution module for lightweight neural networks. It uses an inverted residual structure, which can improve the representation ability of the model while maintaining high efficiency compared to traditional residual connections. It is achieved by applying lightweight depth-wise separable convolutions on the channel dimension of the input feature map, and controlling the number of intermediate channels to achieve channel expansion and compression. This helps improve the representation ability of the model and reduces the computational cost to a certain extent.

[0069] Specifically, MPDIoU is calculated based on the minimum point distance of the horizontal rectangle, which can comprehensively consider the overlapping area, center point distance, and width and height deviations, making the calculation result more accurate.

[0070] Specifically, the improved yolov8-pose network model can not only quickly detect the weeds in the image, but also accurately locate the apical meristem of the weeds. When annotating images, not only the positions of the weeds and crops in the image are annotated, but also the key parts of the weeds, that is, the apical meristem, are marked, and the labeled images are used for training.

[0071] Specifically, Figure 6 , Figure 7 As shown in the figure, in the prior art, the key part of the weed is usually detected by taking the center of the contour as the key part of the weed, which can easily lead to the wrong prediction of the position of the weed apical meristem. In the present invention, it can be clearly seen from the figure that when dealing with various postures of weeds and the interference of ambient light, the prediction result is more accurate and the pixel coordinates of the weed apical meristem in the picture can be more accurately located.

[0072] Specifically, by predicting the location of key parts of weeds, technical support can be provided for the automated precision agriculture industry, for example, for application scenarios such as laser weeding.

[0073] In some embodiments, the labeling and data enhancement of the weed and crop images in the first weed image dataset includes:

[0074] The weed and crop images are processed by changing the image hue, changing the image brightness, rotating the images, and performing mosaic occlusion processing to obtain a second weed image data set after enhancement processing.

[0075] In this embodiment, data enhancement is performed on the weed and crop images in the first data set, so that the data set is expanded to a certain extent. The specific methods of data enhancement are changing the image hue, changing the image brightness, rotating the image, mosaic occlusion and other data enhancement methods. Among them, each image in the data set performs at least one of the data enhancement operations.

[0076] Specifically, for example, a first data set obtained by photographic, video and other equipment contains 2,000 pictures of weeds and crops, and then at least one data enhancement method such as changing image hue, changing image brightness, rotating the picture, mosaic occlusion, etc. is performed on each picture of weeds and crops, and the obtained second weed image data set may contain 4,000 pictures of weeds and crops.

[0077] Specifically, when annotating images, not only the positions of weeds and crops in the images are annotated, but also the key parts of the weeds, that is, the apical meristem, are marked. The labeled images are used for data augmentation and then trained in the improved yolov8-pose network.

[0078] In some embodiments, the MBConv layer adopts an inverted residual structure to improve the representation ability of the model and reduce the computational cost, wherein:

[0079] The computational cost of the depth-separable convolution kernel of the MBConV layer is:

[0080] ;

[0081] Among them, h is the feature map height, w is the feature map width, C out is the number of output channels, k is the convolution kernel size, and n is the multiplication factor.

[0082] In this embodiment, the MBConv layer adopts an inverted residual structure, which can improve the representation ability of the model while maintaining high efficiency compared to the traditional residual connection.

[0083] Specifically, the MBConv layer is implemented by applying lightweight depth-wise separable convolution on the channel dimension of the input feature map, controlling the number of intermediate channels to achieve channel expansion and compression. This helps improve the representation ability of the model and reduces the computational cost to a certain extent.

[0084] Specifically, when the input feature map is , the output feature map remains unchanged, the convolution kernel size is 3, and the computational complexity of the ordinary convolution in the original bottlenet is:

[0085] ;

[0086] Among them, h is the feature map height, w is the feature map width, C out is the number of output channels, C in is the number of input channels, and k is the convolution kernel size.

[0087] The computational cost of the depth-separable convolution kernel of the MBConv layer is:

[0088] ;

[0089] Among them, h is the feature map height, w is the feature map width, C out is the number of output channels, k is the convolution kernel size, and n is the multiplication factor.

[0090] In some embodiments, the calculation formula of the MPDIoU is:

[0091] ;

[0092] ;

[0093] ;

[0094] in, is the x coordinate of the upper left point of the real target box, To predict the x coordinate of the upper left point of the target box, is the y coordinate of the upper left point of the true value target box, is the y coordinate of the upper left point of the predicted value target box, is the x coordinate of the lower right point of the real target box, To predict the x coordinate of the lower right point of the target box, is the y coordinate of the lower right point of the true value target box, is the y coordinate of the lower right point of the predicted target box, d1 is the distance between the upper left point of the real target box and the predicted target box, d2 is the distance between the lower right point of the real target box and the predicted target box, w is the width of the real target box, and h is the height of the real target box. It is the intersection-over-union ratio of the true target box and the predicted target box.

[0095] In this embodiment, the calculation formula of MPDIoU is proposed as follows:

[0096] ;

[0097] ;

[0098] .

[0099] Specifically, factors such as non-overlapping area, center point distance, width and height deviation considered in the loss function can be determined by the coordinates of the upper left point and the lower right point, which shows that MPDIoU greatly simplifies the calculation process and improves the accuracy of bounding box regression.

[0100] In some embodiments, the improved yolov8-pose network model includes: a backbone network, a neck network and a head network; wherein,

[0101] The backbone network includes a first Conv layer, a second Conv layer, a first C2f_EMBC layer, a third Conv layer, a second C2f_EMBC layer, a fourth Conv layer, a third C2f_EMBC layer, a fifth Conv layer, a fourth C2f_EMBC layer and a SPFF layer connected in sequence;

[0102] The neck network includes a first Upsample layer, a first Concat layer, a fifth C2f_EMBC layer, a second Upsample layer, a second Concat layer, a sixth C2f_EMBC layer, a sixth Conv layer, a third Concat layer, a seventh C2f_EMBC layer, a seventh Conv layer, a fourth Concat layer and an eighth C2f_EMBC layer connected in sequence, the first Concat layer can receive the output of the third C2f_EMBC layer for splicing, and the second Concat layer can receive the output of the second C2f_EMBC layer for splicing;

[0103] The head network includes: a first Detect layer, a first Pose layer, a second Detect layer, a second Pose layer, a third Detect layer and a third Pose layer, the first Detect layer and the first Pose layer are connected to the sixth C2f_EMBC layer, the second Detect layer and the second Pose layer are connected to the seventh C2f_EMBC layer, the third Detect layer and the third Pose layer are connected to the eighth C2f_EMBC layer, and the third Concat layer can receive the output of the fifth C2f_EMBC layer for splicing.

[0104] In this embodiment, if Figure 3 As shown, the improved yolov8 network model is introduced, including the backbone network, the neck network and the head network.

[0105] Specifically, the backbone network includes: multiple Conv layers, multiple C2f_EMBC layers and SPFF layers. The Conv layer is used to extract features and compress the size of the weed and crop images, the C2f_EMBC layer is used to fuse feature maps, and the SPFF layer is used to capture multi-scale features of the weed and crop images and fuse the multi-scale features.

[0106] Specifically, the neck network includes: multiple Conv layers, multiple C2f_EMBC layers, multiple Concat layers and multiple Upsample layers. The Upsample layer is used for upsampling operation, that is, increasing the size of the feature map, and the Concat layer is used for splicing the feature map.

[0107] Specifically, the head network includes: multiple Detect layers and multiple Pose layers, wherein the Detect layer is used to process the feature map output by the model, generate bounding boxes, and perform confidence scores on these bounding boxes to determine whether they contain the target object and the position and size of the target object; the Pose layer is used for posture estimation, detecting the apical meristem of the weeds, and estimating the position of the apical meristem of the weeds.

[0108] Specifically, in the backbone network part, for example, the feature map of the 640×640×3 image size is first passed through the first and second Conv layers in sequence to obtain a feature map of 160×160×128 size, and then passed through the first C2f_EMBC layer for further feature extraction and fusion, and then passed through the third Conv layer to obtain a feature map of 80×80×256 size, and then after passing through the second C2f_EMBC layer, the feature map of 80×80×256 size is used as the input of the second Concat and fourth Conv layers, and the feature map of 40×40×512 size output by the fourth Conv layer is used as the input of the third C2f_EMBC layer for feature extraction and fusion, and then the feature map of 40×40×512 size is used as the input of the first Concat layer and the fifth Conv layer. The fifth Conv layer outputs a feature map of 20×20×1024 size, which is then input into the input of the fourth C2f_EMBC layer for feature extraction and fusion, and finally input into the SPFF layer.

[0109] Specifically, in the neck network part, for example: the first Upsample layer that follows enlarges the feature map of 20×20×1024 to a feature map of 40×40×512, inputs it to the first Concat layer and concatenates it with the feature map of 40×40×512 output by the third C2f_EMBC layer, and then inputs it to the fifth C2f_EMBC layer. The second Upsample layer enlarges the feature map of 40×40×512 output by the fifth C2f_EMBC layer to a feature map of 80×80×256, inputs it to the second Concat layer and concatenates it with the feature map of 80×80×256 output by the second C2f_EMBC layer. The images are spliced ​​and then input into the sixth C2f_EMBC layer, followed by input into the sixth Conv layer to obtain a feature map of 40×40×512. The third Concat layer splices the feature map of 40×40×512 output by the sixth Conv layer and the feature map of 40×40×512 output by the fifth C2f_EMBC layer and inputs them into the seventh C2f_EMBC layer for feature extraction and fusion. Subsequently, the seventh Conv layer compresses the feature map of 40×40×512 output by the seventh C2f_EMBC layer into a feature map of 20×20×1024 and inputs it into the fourth Concat, and finally inputs it into the eighth C2f_EMBC layer.

[0110] Specifically, in the head network, for example: the first Detect layer and the first Pose layer take the 80×80×256 feature map output by the sixth C2f_EMBC layer as input, the second Detect layer and the second Pose layer take the 40×40×512 feature map output by the seventh C2f_EMBC layer as input, and the third Detect layer and the third Pose layer take the 20×20×1024 feature map output by the eighth C2f_EMBC layer and the SPPF layer as input to obtain the final result, that is, the pixel coordinates of the apical meristem of the weed.

[0111] Specifically, there is a pose detection head in the head network of the yolov8-pose network model, which is used to detect the prediction of the key points of weeds, so that the model can predict the pixel coordinates of the apical meristem of the weeds.

[0112] In some embodiments, replacing the Bottleneck layer in the C2f convolution block with the MBConV layer to obtain the C2f_EMBC layer further includes: replacing the SE layer in the MBConV layer with the EfficientSE layer;

[0113] The calculation formula of the EfficientSE layer is:

[0114] ;

[0115] Among them, Nc is the attention function, F is the feature map, σ is the activation function, W0 is the fully connected network, is average pooling, is the maximum pooling.

[0116] In this embodiment, the SE layer in the MBConV layer is replaced by an EfficientSE layer.

[0117] Specifically, compared to the EfficientSE attention mechanism, the SE attention mechanism has a disadvantage: the loss of channel information due to the reduction of dimension. In order to avoid the computational burden of such a large model, the two fully connected layers of SE need to reduce the channel dimension. In particular, when the first fully connected layer uses r (scaling channel scale parameter) to reduce the input feature channels and change the number of channels from c to c / r, the second FC layer needs to expand the reduced number of channels to the original channel c. In this process, the reduction of channel dimension leads to the loss of channel information.

[0118] Specifically, the calculation formula of the SE channel attention mechanism is:

[0119] ;

[0120] Among them, Nc is the attention function, F is the feature map, σ is the activation function, W0 and W1 are fully connected networks, is average pooling.

[0121] EfficientSE only replaces two fully connected layers with one fully connected layer with the same number of channels, thus avoiding the loss of channel information. However, EfficientSE compresses the space by only taking the average value of each channel, which may cause some detailed features to be ignored. Therefore, the EfficientSE in the present invention adds the average pooling value of each channel to the maximum pooling value of each channel, which can fuse more information of the image. The calculation formula of the improved EfficientSE attention mechanism is:

[0122] ;

[0123] Among them, Nc is the attention function, F is the feature map, σ is the activation function, W0 is the fully connected network, is average pooling, is the maximum pooling.

[0124] In some embodiments, the network structure of the C2f_EMBC layer includes: a ninth Conv layer, a Split layer, at least two MBConv layers, a fifth Concat layer, and a tenth Conv layer connected in sequence; wherein,

[0125] The network structure of the MBConv layer includes: a first convolutional layer, a depth convolutional layer, an eleventh Conv layer, an efficientSE layer, a second convolutional layer and a Dropout layer connected in sequence.

[0126] In this embodiment, if Figure 4 As shown in the figure, the network structure of the C2f_EMBC layer is introduced, including: Conv layer, Split layer, MBConv layer and Concat layer. Among them, the network structure of the MBConv layer includes: Convolution layer, deep convolution layer, Conv layer, efficientSE layer and Dropout layer.

[0127] Specifically, in the network structure of the C2f_EMBC layer, the Split layer is used to split the feature map output by the ninth Conv layer, and split the feature vector into two parts, one for subsequent convolution and the other for concatenation in the fifth Concat layer. The MBConv layer is used to improve the representation capability of the model, and the fifth Concat layer can directly receive the output of the Split layer and multiple MBConv layers for concatenation of feature maps;

[0128] Specifically, in the network structure of the MBConv layer, the first convolution layer is used to reduce the number of channels of the feature map, the deep convolution layer is used to perform convolution operations on the reduced number of channels, the second convolution layer is used to restore the number of channels, the efficientSE layer is used to add the average pooling value of each channel to the maximum pooling value of each channel, and the Dropout layer is used to prevent the model from overfitting and improve the generalization ability of the model.

[0129] Specifically, the improvement of the C2f_EMBC layer helps to improve the representation ability of the model, reduce the computational cost to a certain extent, and make the location of the apical meristem of weeds more accurate and faster.

[0130] Second, as Figure 2 As shown, another embodiment of the present invention provides a weed key position positioning device, comprising:

[0131] Image acquisition module: used to acquire a first weed image data set and acquire an image of weeds to be detected;

[0132] A data enhancement module: used for labeling and data enhancement of the weed and crop images in the first weed image dataset to obtain a second weed image dataset;

[0133] Model training module: used for building an improved yolov8-pose network model, and training the improved yolov8-pose network model with the second weed image data set to obtain a trained yolov8-pose network model;

[0134] Target detection module: used to detect the weed image to be detected through the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the weed.

[0135] In a third aspect, another embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores a program that can be run on the processor, and when the program is executed by the processor, the method for locating key parts of weeds described in the first aspect is implemented.

[0136] In a fourth aspect, another embodiment of the present invention provides a storage medium having a computer program stored thereon, and when the computer program is executed by at least one processor, the method for locating key parts of weeds described in the first aspect is implemented.

[0137] In summary, the method for locating the key parts of weeds proposed in the present invention can effectively locate the apical meristem of weeds, especially on some weeds with irregular shapes. Compared with the method of obtaining the weed outline by color extraction and then taking the center of the crop outline as the key part of the weed, the positioning accuracy is high and the anti-interference ability is strong. In the case of changes in ambient light, the method for locating the key parts of weeds proposed in the present invention can also have good performance.

[0138] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

[0139] In addition, it should be understood that although the present specification is described according to embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that those skilled in the art can understand.

Claims

1. A method for locating key parts of weeds, characterized in that: The steps include: Acquire a first weed image dataset; Annotating and data enhancing the weed and crop images in the first weed image dataset to obtain a second weed image dataset; Building an improved yolov8-pose network model, and training the improved yolov8-pose network model using the second weed image dataset to obtain a trained yolov8-pose network model; Acquire a weed image to be detected; Inputting the image of the weeds to be detected into the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the weeds; Among them, the improved parts of the improved yolov8-pose network model include: Replace the Bottleneck layer in the C2f convolutional block with the MBConV layer to obtain the C2f_EMBC layer; Replace the loss function with MPDIoU; The improved yolov8-pose network model includes: a backbone network, a neck network and a head network; wherein, The backbone network includes a first Conv layer, a second Conv layer, a first C2f_EMBC layer, a third Conv layer, a second C2f_EMBC layer, a fourth Conv layer, a third C2f_EMBC layer, a fifth Conv layer, a fourth C2f_EMBC layer and a SPFF layer connected in sequence; The neck network includes a first Upsample layer, a first Concat layer, a fifth C2f_EMBC layer, a second Upsample layer, a second Concat layer, a sixth C2f_EMBC layer, a sixth Conv layer, a third Concat layer, a seventh C2f_EMBC layer, a seventh Conv layer, a fourth Concat layer and an eighth C2f_EMBC layer connected in sequence, the first Concat layer receives the output of the third C2f_EMBC layer for splicing, and the second Concat layer receives the output of the second C2f_EMBC layer for splicing; The head network includes: a first Detect layer, a first Pose layer, a second Detect layer, a second Pose layer, a third Detect layer and a third Pose layer, the first Detect layer and the first Pose layer are connected to the sixth C2f_EMBC layer, the second Detect layer and the second Pose layer are connected to the seventh C2f_EMBC layer, the third Detect layer and the third Pose layer are connected to the eighth C2f_EMBC layer, and the third Concat layer receives the output of the fifth C2f_EMBC layer for splicing; The network structure of the C2f_EMBC layer includes: a ninth Conv layer, a Split layer, an MBConv layer, a fifth Concat layer and a tenth Conv layer connected in sequence; wherein, The network structure of the MBConv layer includes: a convolutional layer, a depth convolutional layer, an eleventh Conv layer, an EfficientSE layer, a convolutional layer and a Dropout layer connected in sequence; The calculation formula of the EfficientSE layer is: ； Among them, Nc is the attention function, F is the feature map, σ is the activation function, W0 is the fully connected network, is average pooling, is the maximum pooling.

2. A method for locating key weed parts as claimed in claim 1, characterized in that: The labeling and data enhancement of the weed and crop images in the first weed image dataset includes: The weed and crop images are processed by changing the image hue, changing the image brightness, rotating the images, and performing mosaic occlusion processing to obtain a second weed image data set after enhancement processing.

3. A method for locating key weed parts as claimed in claim 1, characterized in that: The MBConV layer adopts an inverted residual structure to improve the representation ability of the model and reduce the computational cost, where: The computational cost of the depth-separable convolution kernel of the MBConV layer is: ； Among them, h is the feature map height, w is the feature map width, C out is the number of output channels, k is the convolution kernel size, and n is the multiplication factor.

4. A method for locating key weed parts as claimed in claim 1, characterized in that: The calculation formula of the MPDIoU is: ； ； ； in, is the x coordinate of the upper left point of the real target box, To predict the x coordinate of the upper left point of the target box, is the y coordinate of the upper left point of the true value target box, is the y coordinate of the upper left point of the predicted value target box, is the x coordinate of the lower right point of the real target box, To predict the x coordinate of the lower right point of the target box, is the y coordinate of the lower right point of the true value target box, is the y coordinate of the lower right point of the predicted target box, d1 is the distance between the upper left point of the real target box and the predicted target box, d2 is the distance between the lower right point of the real target box and the predicted target box, w is the width of the real target box, and h is the height of the real target box. It is the intersection-over-union ratio of the true target box and the predicted target box.

5. A weed key position positioning device, characterized in that: The device comprises: Image acquisition module: used to acquire a first weed image data set and acquire an image of weeds to be detected; A data enhancement module: used for labeling and data enhancement of the weed and crop images in the first weed image dataset to obtain a second weed image dataset; Model training module: used for building an improved yolov8-pose network model, and training the improved yolov8-pose network model with the second weed image data set to obtain a trained yolov8-pose network model; Target detection module: used to detect the weed image to be detected through the trained yolov8-pose network model to obtain the pixel coordinates of the apical meristem of the weed; The improved yolov8-pose network model includes: a backbone network, a neck network and a head network; wherein The backbone network includes a first Conv layer, a second Conv layer, a first C2f_EMBC layer, a third Conv layer, a second C2f_EMBC layer, a fourth Conv layer, a third C2f_EMBC layer, a fifth Conv layer, a fourth C2f_EMBC layer and a SPFF layer connected in sequence; The neck network includes a first Upsample layer, a first Concat layer, a fifth C2f_EMBC layer, a second Upsample layer, a second Concat layer, a sixth C2f_EMBC layer, a sixth Conv layer, a third Concat layer, a seventh C2f_EMBC layer, a seventh Conv layer, a fourth Concat layer and an eighth C2f_EMBC layer connected in sequence, the first Concat layer receives the output of the third C2f_EMBC layer for splicing, and the second Concat layer receives the output of the second C2f_EMBC layer for splicing; The head network includes: a first Detect layer, a first Pose layer, a second Detect layer, a second Pose layer, a third Detect layer and a third Pose layer, the first Detect layer and the first Pose layer are connected to the sixth C2f_EMBC layer, the second Detect layer and the second Pose layer are connected to the seventh C2f_EMBC layer, the third Detect layer and the third Pose layer are connected to the eighth C2f_EMBC layer, and the third Concat layer receives the output of the fifth C2f_EMBC layer for splicing; The network structure of the C2f_EMBC layer includes: a ninth Conv layer, a Split layer, an MBConv layer, a fifth Concat layer and a tenth Conv layer connected in sequence; wherein, The network structure of the MBConv layer includes: a convolutional layer, a depth convolutional layer, an eleventh Conv layer, an EfficientSE layer, a convolutional layer and a Dropout layer connected in sequence; The calculation formula of the EfficientSE layer is: ； Among them, Nc is the attention function, F is the feature map, σ is the activation function, W0 is the fully connected network, is average pooling, is the maximum pooling.

6. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program that can be run on the processor, and when the program is executed by the processor, the method for locating key parts of weeds according to any one of claims 1 to 4 is implemented.

7. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by at least one processor, the method for locating key parts of weeds according to any one of claims 1 to 4 is implemented.

Images