Peanut legume DUS character quantification method based on key point detection

Abstract

The invention provides a peanut legume DUS character quantification method based on key point detection. The peanut legume DUS character quantification method comprises a key point detection network model, a key point correction algorithm and a legume DUS character quantification algorithm. The key parts of peanut pods can be accurately positioned, and accurate quantification and automatic measurement of main DUS traits of pods of new peanut varieties are realized. Particularly, in a key point detection network model, feature adaptive extraction is realized by introducing deformable convolution into a backbone network, and a lightweight upsampling operator is introduced into a neck network to reserve semantic and spatial information of multiple layers of features, so that the model performance is improved. In the quantification of the key DUS character of the legume, the key DUS character of the legume is measured through a quantification algorithm, including the obvious degree of a legume mouth and the constriction degree of a legume waist, so that the automatic quantification and evaluation of the DUS character of the peanut legume are realized, the phenotypic character difference between different varieties is revealed, and an effective scheme is provided for the identification of new peanut varieties.

Description

Technical Field

[0001] The invention belongs to the technical field of visual detection for identification of new agricultural varieties, and in particular relates to a method for quantifying DUS traits of peanut pods based on key point detection. Background Art

[0002] In terms of DUS trait testing of peanut pods, some domestic studies have focused on the trait identification of peanut pods, but they mainly focus on single traits in peanut pods and have not achieved integrated quantitative display. DUS testing is an evaluation of the specificity, consistency and stability of new plant varieties, and the test of peanut pods is an important part of this evaluation system. Some studies have attempted to use image processing technology, machine learning and other methods to develop automated peanut trait identification and quantification systems, but they are still in the preliminary stage. Some studies have conducted some relevant quantitative analysis on other crops, but compared with other crops, the automatic extraction and quantification of the appearance characteristics of peanut pods, such as surface texture, fruit mouth, fruit waist, etc., has not yet been achieved. These characteristics may be of great significance for the distinction of peanut varieties and the selection of high-quality varieties, providing a scientific basis for breeders. Therefore, future research can consider combining more appearance characteristics for comprehensive analysis.

[0003] In recent years, the effectiveness of deep learning algorithms in key point detection has also been widely verified. Generally speaking, key point detection technology based on deep learning can be divided into two categories: end-to-end methods based on regression and staged methods based on segmentation. Among them, regression methods represented by PoseNet and HRNet can achieve end-to-end key point detection with high real-time performance and accuracy; while staged methods represented by OpenPose can provide more detailed key point information through multi-stage key point positioning and optimization processes, but the computational overhead is large and is not suitable for real-time application scenarios.

[0004] As a newer version of the YOLO series, YOLOv8-pose has been greatly improved in both accuracy and speed. It has shown strong detection performance in commonly used key point detection networks and has a strong advantage in rapid model deployment. However, the effect of directly using the YOLOv8-pose model is not ideal. Summary of the invention

[0005] In view of the above problems, the present invention provides a method for quantifying DUS traits of peanut pods based on key point detection, which includes the following process: S1, obtain peanut pod image; S2, inputting the peanut pod image into the trained key point detection model, and outputting the peanut pod variety and key point positions; The key point detection model is improved based on YOLOv8-pose, including a backbone network, a neck network and a detection head; The backbone network is used to extract features from peanut pod images, and gradually convert the input image into a feature map containing rich semantic information, including a C2f-FasterBlock module and a pyramid SPPELAN module; the C2f-FasterBlock module is used to enhance the detail information of the edge area of ​​the peanut pod; the pyramid SPPELAN module combines multiple pooling layers and convolutional layers to extract multi-scale features; The neck network is used to further fuse and process the multi-scale features extracted from the backbone network, including a C2f-MSDA module and a CAREFE module; the C2f-MSDA module is used to efficiently fuse the multi-scale features extracted from the backbone network to reduce the complexity of the model; the CAREFE module better retains local detail information through content-aware reorganization and kernel prediction mechanisms; The detection head uses the Pose-DBB module to process the input feature map to locate key points in the peanut pod image; S3, an image correction module constructed based on an image processing method, correcting the obtained peanut pod key point data to obtain corrected final key point data; S4, quantification of key DUS traits of peanut pods, including the degree of pod waist constriction and the degree of pod mouth visibility.

[0006] Preferably, when preparing a training data set, the key point detection model places two sides of peanut pods of different varieties on a scanner for scanning in an indoor environment; and annotates the varieties and key points of the dataset images, annotating them as "one frame and six points"; "one frame" refers to the entire peanut pod; "six points" refer to the six key points in the peanut pod, namely the top and bottom points of the peanut pod, the two concave points of the fruit waist, and the top and concave points of the fruit mouth.

[0007] Preferably, the C2f-FasterBlock module in the backbone network utilizes a combination of partial convolution and standard convolution, and uses a ReLU activation function to increase nonlinearity; the processing process is as follows: The feature map is processed in multiple parallel processes:

[0008] Among them, X is the input feature map, Y i is the output feature map of the i-th branch, * represents the convolution operation, W i Represents the convolution kernel of the i-th branch.

[0009] Feature Fusion:

[0010] Among them, Z is the fused feature map, is the weight of each branch output, It is the feature map after convolution.

[0011] Residual Connection:

[0012] Among them, Z′ is the final output feature map obtained through residual connection.

[0013] Preferably, the pyramid SPPELAN module in the backbone network is a convolutional module, which combines multiple pooling layers and convolutional layers to extract multi-scale features; the module extracts features of different levels through the maximum pooling operation, concatenates them, and finally integrates the features through a convolutional layer; the processing process is: Standard convolution Output:

[0014] Among them, W is the convolution kernel and X is the input feature map; Depth Convolution D:

[0015] Among them, W d is the depthwise convolution kernel, and ⨀ is the depthwise convolution operation.

[0016] Point-wise convolution Y:

[0017] Among them, W p It is a 1*1 point-by-point convolution kernel.

[0018] Preferably, the C2f-MSDA module in the neck network first decomposes the standard convolution into depth convolution and point-by-point convolution through depthwise separable convolution DWConv; batch normalization BN accelerates network training and stabilizes the convergence of the model; the scale-aware hole convolution attention mechanism SWDA in the C2f-MSDA module captures multi-scale information through different expansion rates, and uses the attention mechanism to enhance the expression of key features: the processing process is as follows: Multi-scale feature extraction: MSDA applies multi-scale dilated convolution to the input feature map. The dilated convolution produces multiple different receptive fields by controlling the dilation rate:

[0019] Among them, X i Indicates the use of different expansion rates The multi-scale features extracted by the dilated convolution; X is the input feature map; DilatedConv represents the operation of dilated convolution; Combination of dilated convolution and attention mechanism: The dilated convolution provides multi-scale features, and the attention mechanism is used to adaptively assign the importance of each scale:

[0020] in, α 1 ，α 2 ，α 3 represents the weight learned through the attention mechanism; Final output:

[0021] Among them, Y represents the final output, a i Indicates different layers i The weight coefficient of .

[0022] Preferably, the CAREFE module in the neck network comprises a content-aware reorganization module and a core prediction module, wherein the core prediction module is composed of three submodules: a channel compressor, a content encoder and a core normalizer; For the shape The input image is compressed to the number of channels using 1×1 convolution. ; For the compressed input feature map, use The convolutional layer predicts the upsampling kernel, the number of input channels is Cm, and the number of output channels is , expand the channel dimension in the space dimension, and get the shape The upsampling kernel is normalized using softmax so that the sum of the convolution kernel weights is 1. The details are as follows: Input feature map X:

[0023] C is the number of channels, H is the height, W is the width, and R represents the real number space; Generate dynamic convolution kernel W:

[0024] Conv(x) represents the convolution operation; Softmax() is the softmax activation function; To perform a reorganization operation:

[0025] i, j Represents the coordinates in the reorganized feature map Y; m represents the element index of the convolution kernel; Indicates that in the dynamic convolution kernel W, the position ( i, j ) is the weight of the mth element; Indicates that in the input feature map X, channel c and position ( i, j ) on the eigenvalue; k 2 Indicates the total number of elements of the convolution kernel; Output feature map Y.

[0026] Preferably, the detection head adopts the Pose-DBB module, which processes different parts of the input feature map in parallel through a multi-branch convolution structure; its working mechanism is as follows: (1) Input feature map X:

[0027] Among them, C is the number of channels, H is the height, and W is the width; (2) Convolution branch: 3×3 convolution branch Y 3×3 :

[0028] Among them, Conv 3×3 (X) represents a 3×3 convolution operation; 1×1 convolution branch Y 1×1 :

[0029] Among them, Conv 1×1 (X) represents a 1×1 convolution operation; Dilated convolution Y dilated :

[0030] Among them, DilatedConv 3×3 (X) represents a 3×3 dilated convolution operation; Global average pooling branch Y GAP :

[0031] Where GAP(X) represents the global average operation; (3) Output feature map merging: The outputs of each branch are summed in the channel dimension to obtain the final output Y:

[0032] Among them, Y zero Represents a zero-filled feature map (4) Structural reparameterization:

[0033] in, It is a single convolution kernel obtained by merging the convolution kernels of each branch, which is used to accelerate the reasoning process.

[0034] Preferably, the specific processing process of the image correction module is: S31, converting all peanut pod RGB images into grayscale images, and then converting the grayscale images into binary images using a thresholding method; S32, using morphological operations to open the image; through multiple erosion and expansion operations, the noise in the image is removed and the main features are highlighted; S33, using edge detection Sobel algorithm to detect the edge contour of peanut pods; extracting edge information in the image by calculating the gradients in the X and Y directions and combining their amplitudes; then analyzing the connected areas of the image, filtering out larger image contours and filtering out smaller contour images; Calculation of image contours: For the peanut pod image, calculate the gradient and :

[0035]

[0036] in, and Respectively horizontal and vertical gradients in position; The image is The pixel value at is the value of the Sobel horizontal convolution kernel; is the value of the Sobel vertical convolution kernel;

[0037]

[0038] Among them, Edge strength, is the edge direction; According to the gradient amplitude , apply a threshold T:

[0039] in is a binary image, T is the threshold; Use connected domain analysis to mark each connected domain in the image; and select the larger connected domain as the required contour; the specific algorithm is as follows:

[0040] in, is the final connected component label for each pixel in the image; is the connected domain label of the pixel, is the marker of the largest connected domain, is the background tag; S34, determining the centroid of the contour by calculating the image moment as a reference for the center; determining the position of the key point relative to the centroid according to the key point deviation calculation, and performing key point correction; the key point deviation calculation process: Calculate the centroid of the contour :

[0041]

[0042] in, are the coordinates of the contour points; is the mass of the point; is the number of points of the contour; For each key point , calculate its centroid Position relationship:

[0043] in, is the deviation of the keypoint from the centroid.

[0044] Preferably, the quantification of the degree of constriction of the waist of the pod is specifically as follows: The distance between the two key points w-point1 and w-point2 at the waist of the fruit is used to quantify the degree of constriction at the waist of the fruit. The specific calculation process is as follows: First, according to the detection of the network model, the coordinates of w-point1 and w-point2 are obtained, which are and , then the horizontal distance w of the key point at the waist is:

[0045]

[0046] Among them, CPI is the fruit waist shrinkage index, W is the maximum horizontal width of the peanut pod; w is the horizontal distance between the two key points at the fruit waist, the smaller the CPI, the greater the degree of shrinkage of the peanut pod; the peanuts are graded according to the calculated fruit waist shrinkage index CPI.

[0047] Preferably, the quantification of the degree of fruit mouth visibility of the pod is specifically as follows: According to the convex point m-point1 and concave point m-point2 The relative position and angle of are quantified and divided into the following three cases: (1) Normally, the convex point of the fruit mouth is above the concave point and the angle is less than 90 degrees; (2) The convex point of the fruit mouth is below the concave point; (3) The convex point of the fruit mouth is above the concave point but the angle is greater than 90 degrees; The specific calculation method is as follows: In case (1), the fruit mouth is relatively normal, and the convex point is higher than the concave point, indicating that the fruit mouth is more significant; the specific quantification method is as follows:

[0048] in, It is the key point m-point1 and m-point2 The angle formed; In case (2), the shape of the fruit mouth is abnormal, with the convex point located below the concave point, indicating that the fruit mouth is not obvious or even reversed, and is directly judged as abnormal without quantification;

[0049] In case (3), the convex point is above the concave point, but the angle is greater than 90 degrees: due to the larger angle, the mouth of the fruit appears blunt or flat:

[0050] According to the above three situations, the angle The size of the fruit mouth is divided into the following four categories: Level 1: , the fruit mouth is obvious; Level 2: , the fruit mouth is obviously reduced; Level 3: , the fruit mouth has almost no protrusion; Level 4: , the mouth is not obvious at all.

[0051] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention uses a deep learning algorithm to detect key points of peanut pods, can effectively test the DUS traits of peanut pods, makes a certain contribution to quantifying the traits of peanut pods, and has high practical application value.

[0052] 2. In the backbone network part, the conventional convolution used in the backbone network of the original YOLOv8-Pose has a very limited ability to model geometric transformations. It relies on a fixed geometric structure. The convolution unit samples the input feature map at a fixed position, and the pooling layer reduces the spatial resolution at a fixed ratio, which limits its ability to capture various geometric transformations.

[0053] Due to the differences in the shapes and angles of peanut pods, traditional convolutional networks have limited modeling capabilities for geometric transformations. By introducing the SPPELAN module, the model can better adapt and extract key features when processing peanut pod images of different shapes, sizes, and angles.

[0054] Secondly, since the key point information of peanut pods is located in the edge area of ​​​​the peanut pods, the introduction of the Cf2-FasterBlock module can better enhance the detail information of the edge area of ​​​​the peanut pods and improve the feature extraction ability of the key points of peanut pods.

[0055] 3. In the neck network structure, C2f-MSDA is introduced to efficiently fuse the multi-scale features extracted from the backbone network, reduce the complexity of the model, and solve the problem of less peanut pod training data sets in each category. The key point information of peanut pods is usually located in the edge area of ​​the pod, and there is a problem of unclear features, which leads to a decrease in model efficiency. By introducing the CARAFE module, these local detail information can be better retained, making the model more accurate in key point detection.

[0056] 4. The original detection head of YOLOv8-Pose is a single-path feature extraction method, while Pose-DBB uses a multi-branch convolutional structure to process different parts of the input feature map in parallel. In the process of labeling peanut pods, the number of key points is often inconsistent, which may lead to labeling errors. The introduction of the Pose-DBB module can effectively address this problem. Through its dual-path feature extraction structure, it enhances the accuracy of key point positioning, thereby reducing the impact of inconsistent labeling. At the same time, it can extract more efficient features during training without affecting the speed of inference.

[0057] 5. The present invention has newly designed a key point correction method to improve the accuracy of key points.

[0058] 6. The present invention has newly designed a quantification method to quantify the key DUS traits of peanut pods, including the quantification of the degree of pod waist constriction and the degree of fruit mouth visibility. Combined with the above-mentioned key point detection model, a more mature quantification method is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of the overall scheme of the present invention; Figure 2 The overall structural principle block diagram of the key point detection model of the present invention; Figure 3 It is a detailed diagram of the overall structure of the key point detection model of the present invention; Figure 4 The figure is a workflow diagram of the key point correction model of the present invention.

[0060] Figure 5 This is a diagram showing the correction effect of the method of the present invention at key points in this example.

[0061] Figure 6 Schematic diagram of the method for quantifying the middle and mouth of peanut pods according to the present invention. DETAILED DESCRIPTION

[0062] The present invention is further described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In order to better quantify the appearance phenotype of peanut pods and solve the current problems in phenotypic measurement, the present invention proposes a model for automatic measurement of peanut pod phenotype based on deep learning. The model can be embedded in a variety of devices, suitable for operation on multiple platforms, and effectively realizes the detection of specificity, consistency and stability of new varieties of peanut pods. Compared with the original YOLOv8 model, the newly proposed model can greatly improve the accuracy of key point detection of peanut pods, providing strong support for the precise detection and positioning of quantification platform equipment.

[0064] like Figure 1 As shown, the present invention first collects images of peanut pods, completes data construction and annotation, and then uses the constructed model to train the peanut pod key point detection model, and then implements the correction of the peanut pod key points to complete the quantification of the DUS trait. The implementation process of the present invention is as follows: Step 1: Use a scanner to scan peanut pod images from different angles to collect a data set. Label the data set and divide it into a training set, a validation set, and a test set according to a certain ratio; Step 2, constructing a network model and training the network model to improve the performance of the model, using the peanut pod data constructed in step 2 to train the model; Step 3: Use the image data in the test set to test the trained key point detection model and complete the model construction; Step 4: After the model is built, the key point data is corrected using image processing technology; Step 5, embed the final model structure into the hardware device, so that the quantitative information of the appearance of peanut pods can be displayed on the platform interface of the device to realize the detection of new varieties; This example further illustrates the peanut pod quantitative detection method of the present invention through a detailed experimental process.

[0065] 1. Get the original data The present invention uses a scanner to scan high-definition peanut pod images, including peanut pod images of different types and at different angles.

[0066] 2. Data Preprocessing After image preprocessing of the collected image dataset, including image cutting, filtering and compression, the image annotation tool was used to accurately annotate the tea buds in the image. Then, the scanned image was annotated using the LabelMe tool. The annotated information included the category of peanut pods, the number of the waist of the peanut pods, and the number of the mouth of the peanut pods. The annotated label file was saved in TXT format. A total of six key points were annotated.

[0067] For example, when marking one type of peanut pods, the marking results are: top, bottom, w-point1, w-point2, m-point1, m-point2, where top and bottom are the top and bottom points of the peanut, w-point1 and w-point2 are the two concave points on the waist of the fruit, and m-point1 and m-point2 are the convex point and concave point of the fruit mouth, respectively.

[0068] These labeled data will be used as the original data set. Subsequently, the data set is divided into training set, validation set and test set in a ratio of 8:1:1 for model training and evaluation.

[0069] 3. Model building The YOLOv8_PB model mainly consists of a backbone network, a neck network, and a detection head. Figure 2 and Figure 3As shown. The backbone network mainly includes the C2-FasterBlock module and the pyramid SPPELAN module. The C2-FasterBlock module consists of PConv, batch normalization layer and ReLU activation function layer, aiming to achieve faster and higher model performance. The SPPELAN module fuses local features and global features to enrich the expressiveness of the output feature map. This module extracts features from different levels through the maximum pooling operation, concatenates them, and finally integrates the features through a convolution layer. The neck structure is used to fuse feature maps from different scales to improve the effect of target detection, mainly including the C2f-MSDA module and the CARAFE module. Among them, C2f-MSDA is an improved convolution module that combines depthwise separable convolution (DWConv) and standard convolution, and adds multiple linear layers and GELU activation functions to improve the diversity and expressiveness of the model. The CARAFE module is a module composed of a channel compressor and a normalization kernel, which is used to upsample or downsample feature maps. Its main function is to enhance the fusion effect of multi-scale features. The detection head Pose-DBB is a multi-branch convolutional module that extracts diverse features through different convolution and pooling operations, and finally integrates them to enhance the model's ability to recognize complex targets.

[0070] (1) Backbone network: The conventional convolution used in the backbone network of the original YOLOv8-Pose has a very limited ability to model geometric transformations. It relies on a fixed geometric structure. The convolution unit samples the input feature map at a fixed position, and the pooling layer reduces the spatial resolution at a fixed ratio, which limits its ability to capture various geometric transformations.

[0071] Due to the differences in the shapes and angles of peanut pods, traditional convolutional networks have limited modeling capabilities for geometric transformations. By introducing the SPPELAN module, the model can better adapt and extract key features when processing peanut pod images of different shapes, sizes, and angles.

[0072] Secondly, since the key point information of peanut pods is located in the edge area of ​​​​the peanut pods, the introduction of the Cf2-FasterBlock module can better enhance the detail information of the edge area of ​​​​the peanut pods and improve the feature extraction ability of the key points of peanut pods.

[0073] The Cf2-FasterBlock module utilizes a combination of partial convolution (PConv) and standard convolution, using the ReLU activation function to increase nonlinearity while maintaining high computational efficiency. Its working principle is as follows: 1) The feature map is processed in multiple parallel processes:

[0074] Among them, X is the input feature map, Y i is the output feature map of the i-th branch, * represents the convolution operation, W i Represents the convolution kernel of the i-th branch.

[0075] 2) Feature fusion:

[0076] Among them, Z is the fused feature map, is the weight of each branch output, It is the feature map after convolution.

[0077] 3) Residual connection:

[0078] Among them, Z′ is the final output feature map obtained through residual connection.

[0079] The pyramid SPPELAN module is a convolutional module that combines multiple pooling layers and convolutional layers to extract multi-scale features. The module extracts features at different levels through the maximum pooling operation, concatenates them, and finally integrates the features through a convolutional layer. Its working mechanism is as follows: 1) Standard convolution Output:

[0080] Among them, W is the convolution kernel and X is the input feature map.

[0081] 2) Depth Convolution D:

[0082] Among them, W d is the depthwise convolution kernel, and ⨀ is the depthwise convolution operation.

[0083] 3) Point-wise convolution Y:

[0084] Among them, W p It is a 1*1 point-by-point convolution kernel, and D is the depth convolution.

[0085] (2) The neck network part is used to further fuse and process the multi-scale features extracted from the backbone network to enhance the detection ability of the model. The neck structure is very critical in the target detection model because it is responsible for combining feature maps of different scales so that the model can better identify targets of different sizes. It mainly includes the C2f-MSDA module and the CAREFE module.

[0086] In the neck network structure, C2f-MSDA is introduced to efficiently fuse the multi-scale features extracted from the backbone network, reduce the complexity of the model, and solve the problem of less peanut pod training data sets in each category. The key point information of peanut pods is usually located in the edge area of ​​the pod, and there is a problem of unclear features, which leads to a decrease in model efficiency. By introducing the CARAFE module, these local detail information can be better retained, making the model more accurate in key point detection.

[0087] C2f-MSDA is specifically designed to fuse multi-scale features and enhance the model's perception of different targets. The module first decomposes standard convolution into depthwise convolution and pointwise convolution through depthwise separable convolution (DWConv), which significantly reduces the number of parameters and computation while maintaining the effect of feature extraction. Then, batch normalization (BN) accelerates network training and stabilizes the convergence of the model. The scale-aware hole convolution attention mechanism (SWDA) in the module captures multi-scale information through different expansion rates and uses the attention mechanism to enhance the expression of key features. Its working mechanism is as follows: 1) Multi-scale feature extraction: MSDA applies multi-scale dilated convolution to the input feature map. The dilated convolution produces multiple different receptive fields by controlling the expansion rate, which is equivalent to gradually observing the image features from local to global.

[0088]

[0089] Among them, X i Indicates the use of different expansion rates r i The multi-scale features extracted by the dilated convolution of X; X is the input feature map. DilatedConv represents the operation of dilated convolution.

[0090] 2) Combination of dilated convolution and attention mechanism: The dilated convolution provides multi-scale features, and the attention mechanism is used to adaptively assign the importance of each scale.

[0091]

[0092] Among them, α1, α2α3 represent the weights learned through the attention mechanism.

[0093] 3) Final output:

[0094] Among them, Y represents the final output, a i Represents the weight coefficient of different layers i.

[0095] At the same time, the content-aware feature reorganization operator CARAFE is used for upsampling. CAREFE includes two main components: content-aware reorganization module and kernel prediction module. The kernel prediction module consists of three submodules: channel compressor, content encoder and kernel normalizer. CARAFE has three advantages: first, it has a large acceptance range. It can collect information in a large acceptance field; second, it is highly adaptable and can dynamically adapt to instance-specific content; third, it is lightweight and has high computational efficiency; the working principle of the CARAFE upsampling operator is: for shapes The input image is compressed to the number of channels using 1×1 convolution. ; For the compressed input feature map, use The convolutional layer predicts the upsampling kernel, the number of input channels is Cm, and the number of output channels is , expand the channel dimension in the space dimension, and get the shape The upsampling kernel is normalized by softmax so that the sum of the convolution kernel weights is 1. Its working mechanism is as follows: 1) Input feature map X:

[0096] C is the number of channels, H is the height, W is the width, and R represents the real number space.

[0097] 2) Generate dynamic convolution kernel W:

[0098] Conv(x) represents the convolution operation; Softmax() is the softmax activation function.

[0099] 3) Perform reorganization operations:

[0100] i, j represent the coordinates in the reorganized feature map Y; m represents the element index of the convolution kernel; W m,i,j represents the weight of the mth element at position (i, j) in the dynamic convolution kernel W; X c,i,j represents the feature value at channel c and position (i, j) in the input feature map X; k 2 Indicates the total number of elements in the convolution kernel.

[0101] 4) Output feature map Y.

[0102] In this way, CARAFE can effectively improve the quality of upsampling and enhance the detail expression ability of feature maps.

[0103] (3) Detection head The original detection head of YOLOv8-Pose is a single-path feature extraction method, while Pose-DBB uses a multi-branch convolution structure to process different parts of the input feature map in parallel. In the process of labeling peanut pods, the number of key points is often inconsistent, which may lead to labeling errors. The introduction of the Pose-DBB module can effectively address this problem. Through its dual-path feature extraction structure, it enhances the accuracy of key point positioning, thereby reducing the impact caused by inconsistent labeling. Its working mechanism is as follows: 1) Input feature map X:

[0104] Among them, C is the number of channels, H is the height, and W is the width.

[0105] 2) Convolution branch: 3×3 convolution branch Y 3×3 :

[0106] Among them, Conv 3×3 (X) represents a 3×3 convolution operation.

[0107] 1×1 convolution branch Y 1×1 :

[0108] Among them, Conv 1×1 (X) represents a 1×1 convolution operation.

[0109] Dilated convolution Y dilated :

[0110] Among them, DilatedConv 3×3 (X) represents a 3×3 dilated convolution operation.

[0111] Global average pooling branch Y GAP :

[0112] Here, GAP(X) represents the global average operation.

[0113] 3) Output feature map merging: The outputs of each branch are summed in the channel dimension to obtain the final output Y:

[0114] Among them, Y zero represents a zero-filled feature map; 4) Structural reparameterization:

[0115] in, It is a single convolution kernel obtained by merging the convolution kernels of each branch, which is used to accelerate the reasoning process.

[0116] Through these optimizations, the YOLOv8_PB model significantly improves the accuracy and robustness of key point detection while maintaining efficient computation.

[0117] 4. Model Training Experimental platform: Pytorch deep learning framework, Win10 64 operating system, GPU is RTX 3090 (24GB), CPU is 12th Gen Intel(R) Core(TM) i5-12490F 3.00 GHz, memory is 16G. The number of training images per batch is set to 16, the Epoch is set to 300, the learning rate is initialized to 0.001, and the stochastic gradient descent strategy SGD is used to optimize the network parameters during the training phase.

[0118] 5. Image correction module After the YOLOv8_PB model training is completed, the network model is called. In actual use, the model may have the problem of key point data offset. After the model is tested, it will be fine-tuned. The specific process is as follows: Figure 4 The specific process is as follows: S1, converts all RGB peanut pod images into grayscale images to reduce the complexity of calculation. Then, the thresholding method is used to convert the grayscale images into black and white binary images and enter S2; S2, use morphological operations to open the image. Through 3 corrosion and 2 expansion operations, the noise in the image is removed and the main features are highlighted; S3, using the edge detection Sobel algorithm to detect the edge contour of the peanut pod. By calculating the gradients in the X and Y directions and combining their amplitudes, the edge information in the image is extracted. In order to remove the image contour information of other debris on the image, the connected areas of the image are analyzed, the larger image contours are screened out, and the smaller contour images are filtered out; The specific calculation method is as follows: For the peanut pod image, calculate the gradient and :

[0119]

[0120] in, and Respectively horizontal and vertical gradients in position; The image is The pixel value at is the value of the Sobel horizontal convolution kernel. is the value of the Sobel vertical convolution kernel.

[0121]

[0122]

[0123] Among them, Edge strength, is the edge direction.

[0124] According to the gradient amplitude , apply a threshold T:

[0125] here, is the binary image, and T is the threshold.

[0126] Use connected domain analysis to mark each connected domain in the image. And select the larger connected domain as the required contour. The specific algorithm is as follows:

[0127] in, is the final connected component label for each pixel in the image. is the connected domain label of the pixel, is the marker of the largest connected domain, Is the background tag.

[0128] S4, since the model has been identified in the early stage, verification is performed at the key points identified by the model. The centroid of the contour is determined by calculating the image moment as a reference for the center. For example, if the key point W1 (a key point at the waist of the fruit) is located on the right side of the centroid, then the point with the smallest x-coordinate value closest to W1 is selected on the contour, and this point is within 10 pixels to the right of W1. If the key point W1 is located on the left side of the centroid, then the point with the largest x-coordinate value is selected, and this point is within 10 pixels to the left of W1.

[0129] The specific calculation method is as follows: Calculate the centroid of the contour :

[0130]

[0131] in, are the coordinates of the contour points. is the mass of the point (usually 1). is the number of points of the contour.

[0132] For each key point , calculate its centroid position relationship.

[0133]

[0134] in, is the deviation of the keypoint from the centroid.

[0135] According to the calculated deviation, the position of the key point relative to the center of mass is further determined, so as to select different discrimination methods. The following is still explained by taking the w1 key point as an example.

[0136]

[0137] in, is the x-coordinate of the final corrected point. Indicates that the final key point to be found is on the contour. If the key point W1 (a key point at the waist of the fruit) is on the right side of the centroid, select the point on the contour that is closest to W1 and has the smallest x-coordinate value. This point is within 10 pixels to the right of W1. If the key point W1 is on the left side of the centroid, select the point with the largest x-coordinate value. This point is within 10 pixels to the left of W1.

[0138] The final effect after correction is as follows Figure 5 As shown in the figure, it can be seen that the corrected effect of each key point can reach a relatively ideal level.

[0139] 6. Shape Quantization The final peanut pods need to be further quantified. For specific methods, refer to Figure 6 .

[0140] (1) Degree of constriction of the waist The degree of constriction of the fruit waist refers to the degree of contraction of the middle part of the peanut pod. In order to accurately quantify, the present invention uses the distance between the two key points w-point1 and w-point2 at the fruit waist to quantify the degree of constriction at the fruit waist. The specific calculation formula is as follows.

[0141] First, according to the detection of the network model, the coordinates of w-point1 and w-point2 can be obtained, which are and , then the horizontal distance w of the key point at the waist is:

[0142]

[0143] Among them, CPI is the fruit waist shrinkage index, W is the maximum horizontal width of the peanut pod, and w is the horizontal distance between the two key points at the fruit waist. The smaller the CPI, the greater the degree of peanut pod shrinkage.

[0144] According to the calculated fruit waist shrinkage index (CPI), it can be divided into 4 levels, and each level is described in detail to better express its meaning.

[0145] 1) The waist of the peanut pod shrinks significantly, and the shape shows a distinct waist-shrinking state.

[0146] 2) The peanut pod waist becomes narrower and shrinks significantly. The pod shape shows a certain bottleneck shape.

[0147] 3) The peanut pod waist shrinks less. The pod shape is more uniform and full.

[0148] 4) , the peanut pod waist has almost no shrinkage phenomenon. The pod shape is relatively complete.

[0149] (2) Prominence of the fruit mouth The degree of fruit mouth visibility refers to the degree of prominence of the protrusion on the top of the peanut pod. and concave point m-point2 The relative position and angle of the quantification can be divided into the following three cases: 1) Normally, the convex point of the fruit mouth is above the concave point and the angle is less than 90 degrees; 2) The convex point of the fruit mouth is below the concave point; 3) The convex point of the fruit mouth is above the concave point but the angle is greater than 90 degrees.

[0150] The specific calculation method is as follows: Case 1): The convex point is above the concave point, and the angle is less than 90 degrees; in this case, the fruit mouth is relatively normal, and the convex point is higher than the concave point, indicating that the fruit mouth is more significant. The specific quantification method is as follows:

[0151] in, It is the key point m-point1 and m-point2 The angle formed.

[0152] Case 2): The convex point is below the concave point: In this case, the shape of the fruit mouth may be abnormal, with the convex point located below the concave point, indicating that the fruit mouth is not obvious and may even be reversed; it is judged as abnormal and does not need to be quantified.

[0153]

[0154] Case 3): The convex point is above the concave point, but the angle is greater than 90 degrees; in this case, the convex point is still higher than the concave point, but due to the larger angle, the fruit mouth appears blunt or flat.

[0155]

[0156] According to the above three wind conditions, the angle According to the size of the fruit mouth, the fruit grade is divided into the following three categories.

[0157] Level 1: , the fruit mouth is obvious Level 2: , the degree of fruit mouth is obviously reduced.

[0158] Level 3: , the fruit mouth has almost no protrusion.

[0159] Level 4: , the mouth is not obvious at all.

[0160] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0161] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for quantifying DUS traits of peanut pods based on key point detection, characterized in that: The process includes: S1, obtain peanut pod image; S2, inputting the peanut pod image into the trained key point detection model, and outputting the peanut pod variety and key point positions; The key point detection model is improved based on YOLOv8-pose, including a backbone network, a neck network and a detection head; The backbone network is used to extract features from peanut pod images, and gradually convert the input image into a feature map containing rich semantic information, including a C2f-FasterBlock module and a pyramid SPPELAN module; the C2f-FasterBlock module is used to enhance the detail information of the edge area of ​​the peanut pod; the pyramid SPPELAN module combines multiple pooling layers and convolutional layers to extract multi-scale features; The neck network is used to further fuse and process the multi-scale features extracted from the backbone network, including a C2f-MSDA module and a CAREFE module; the C2f-MSDA module is used to efficiently fuse the multi-scale features extracted from the backbone network to reduce the complexity of the model; the CAREFE module better retains local detail information through content-aware reorganization and kernel prediction mechanisms; The detection head uses the Pose-DBB module to process the input feature map to locate key points in the peanut pod image; S3, an image correction module constructed based on an image processing method, correcting the obtained peanut pod key point data to obtain corrected final key point data; S4, quantification of key DUS traits of peanut pods, including the degree of pod waist constriction and the degree of pod mouth visibility.

2. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: When preparing the training data set for the key point detection model, the two sides of peanut pods of different varieties are placed on a scanner for scanning in an indoor environment; and the varieties and key points of the dataset images are annotated as "one frame and six points"; "one frame" refers to the entire peanut pod; "six points" refer to the six key points in the peanut pod, namely the top and bottom points of the peanut pod, the two concave points of the fruit waist, and the top and concave points of the fruit mouth.

3. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The C2f-FasterBlock module in the backbone network uses a combination of partial convolution and standard convolution and uses the ReLU activation function to increase nonlinearity; the processing process is as follows: The feature map is processed in multiple parallel processes: Among them, X is the input feature map, Y i is the output feature map of the i-th branch, * represents the convolution operation, W i Represents the convolution kernel of the i-th branch. Feature Fusion: Among them, Z is the fused feature map, is the weight of each branch output, It is the feature map after convolution. Residual Connection: Among them, Z′ is the final output feature map obtained through residual connection.

4. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The pyramid SPPELAN module in the backbone network is a convolutional module that combines multiple pooling layers and convolutional layers to extract multi-scale features. The module extracts features at different levels through a maximum pooling operation, concatenates them, and finally integrates the features through a convolutional layer. The processing process is: Standard convolution Output: Among them, W is the convolution kernel and X is the input feature map; Depth Convolution D: Among them, W d is the depthwise convolution kernel, and ⨀ is the depthwise convolution operation. Point-wise convolution Y: Among them, W p It is a 1*1 point-by-point convolution kernel.

5. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The C2f-MSDA module in the neck network first decomposes the standard convolution into depth convolution and point-by-point convolution through depthwise separable convolution DWConv; batch normalization BN accelerates network training and stabilizes the convergence of the model; the scale-aware hole convolution attention mechanism SWDA in the C2f-MSDA module captures multi-scale information through different expansion rates, and uses the attention mechanism to enhance the expression of key features: the processing process is as follows: Multi-scale feature extraction: MSDA applies multi-scale dilated convolution to the input feature map. The dilated convolution produces multiple different receptive fields by controlling the dilation rate: Among them, X i Indicates the use of different expansion rates The multi-scale features extracted by the dilated convolution; X is the input feature map; DilatedConv represents the operation of dilated convolution; Combination of dilated convolution and attention mechanism: The dilated convolution provides multi-scale features, and the attention mechanism is used to adaptively assign the importance of each scale: in, α 1 ，α 2 ，α 3 represents the weight learned through the attention mechanism; Final output: Among them, Y represents the final output, a i Indicates different layers i The weight coefficient of .

6. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The CAREFE module in the neck network includes a content-aware reorganization module and a kernel prediction module, and the kernel prediction module consists of three submodules: a channel compressor, a content encoder, and a kernel normalizer; For the shape The input image is compressed to the number of channels using 1×1 convolution. ; For the compressed input feature map, use The convolutional layer predicts the upsampling kernel, the number of input channels is Cm, and the number of output channels is , expand the channel dimension in the space dimension, and get the shape The upsampling kernel is normalized using softmax so that the sum of the convolution kernel weights is 1. The details are as follows: Input feature map X: C is the number of channels, H is the height, W is the width, and R represents the real number space; Generate dynamic convolution kernel W: Conv(x) represents the convolution operation; Softmax() is the softmax activation function; To perform a reorganization operation: i, j Represents the coordinates in the reorganized feature map Y; m represents the element index of the convolution kernel; Indicates that in the dynamic convolution kernel W, the position ( i, j ) is the weight of the mth element; Indicates that in the input feature map X, channel c and position ( i, j ) on the eigenvalue; k 2 Indicates the total number of elements of the convolution kernel; Output feature map Y.

7. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The detection head adopts the Pose-DBB module, which processes different parts of the input feature map in parallel through a multi-branch convolution structure; its working mechanism is as follows: (1) Input feature map X: Among them, C is the number of channels, H is the height, and W is the width; (2) Convolution branch: 3×3 convolution branch Y 3×3 : Among them, Conv 3×3 (X) represents a 3×3 convolution operation; 1×1 convolution branch Y 1×1 : Among them, Conv 1×1 (X) represents a 1×1 convolution operation; Dilated convolution Y dilated : Among them, DilatedConv 3×3 (X) represents a 3×3 dilated convolution operation; Global average pooling branch Y GAP : Where GAP(X) represents the global average operation; (3) Output feature map merging: The outputs of each branch are summed in the channel dimension to obtain the final output Y: Among them, Y zero Represents a zero-filled feature map (4) Structural reparameterization: in, It is a single convolution kernel obtained by merging the convolution kernels of each branch, which is used to accelerate the reasoning process.

8. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The specific processing process of the image correction module is as follows: S31, converting all peanut pod RGB images into grayscale images, and then converting the grayscale images into binary images using a thresholding method; S32, using morphological operations to open the image; through multiple erosion and expansion operations, the noise in the image is removed and the main features are highlighted; S33, using edge detection Sobel algorithm to detect the edge contour of peanut pods; extracting edge information in the image by calculating the gradients in the X and Y directions and combining their amplitudes; then analyzing the connected areas of the image, filtering out larger image contours and filtering out smaller contour images; Calculation of image contours: For the peanut pod image, calculate the gradient and : in, and Respectively horizontal and vertical gradients in position; The image is The pixel value at is the value of the Sobel horizontal convolution kernel; is the value of the Sobel vertical convolution kernel; Among them, Edge strength, is the edge direction; According to the gradient amplitude , apply a threshold T: in is a binary image, T is the threshold; Use connected domain analysis to mark each connected domain in the image; and select the larger connected domain as the required contour; the specific algorithm is as follows: in, is the final connected component label for each pixel in the image; is the connected domain label of the pixel, is the marker of the largest connected domain, is the background tag; S34, determining the centroid of the contour by calculating the image moment as a reference for the center; determining the position of the key point relative to the centroid according to the key point deviation calculation, and performing key point correction; the key point deviation calculation process: Calculate the centroid of the contour : in, are the coordinates of the contour points; is the mass of the point; is the number of points of the contour; For each key point , calculate its centroid Position relationship: in, is the deviation of the keypoint from the centroid.

9. A method for quantifying peanut pod DUS traits based on key point detection as claimed in claim 1, characterized in that: The quantification of the degree of constriction of the waist of the pod is specifically as follows: The distance between the two key points w-point1 and w-point2 at the waist of the fruit is used to quantify the degree of constriction at the waist of the fruit. The specific calculation process is as follows: First, according to the detection of the network model, the coordinates of w-point1 and w-point2 are obtained, which are and , then the horizontal distance w of the key point at the waist is: Among them, CPI is the fruit waist shrinkage index, W is the maximum horizontal width of the peanut pod; w is the horizontal distance between the two key points at the fruit waist, the smaller the CPI, the greater the degree of shrinkage of the peanut pod; the peanuts are graded according to the calculated fruit waist shrinkage index CPI.

10. A peanut pod DUS trait quantification method based on key point detection as claimed in claim 1, characterized in that: The quantification of the degree of obviousness of the fruit mouth of the pod is specifically as follows: According to the convex point m-point1 and concave point m-point2 The relative position and angle of are quantified and divided into the following three cases: (1) Normally, the convex point of the fruit mouth is above the concave point and the angle is less than 90 degrees; (2) The convex point of the fruit mouth is below the concave point; (3) The convex point of the fruit mouth is above the concave point but the angle is greater than 90 degrees; The specific calculation method is as follows: In case (1), the fruit mouth is relatively normal, and the convex point is higher than the concave point, indicating that the fruit mouth is more significant; the specific quantification method is as follows: in, It is the key point m-point1 and m-point2 The angle formed; In case (2), the shape of the fruit mouth is abnormal, with the convex point located below the concave point, indicating that the fruit mouth is not obvious or even reversed, and is directly judged as abnormal without quantification; In case (3), the convex point is above the concave point, but the angle is greater than 90 degrees: due to the larger angle, the mouth of the fruit appears blunt or flat: According to the above three situations, the angle The size of the fruit mouth is divided into the following four categories: Level 1: , the fruit mouth is obvious; Level 2: , the fruit mouth is obviously reduced; Level 3: , the fruit mouth has almost no protrusion; Level 4: , the fruit mouth is not obvious at all.

Images