Improved soybean seed testing system and method based on YOLO v8

Abstract

The present invention relates to the technical field of soybean seed testing, and in particular to a soybean seed testing system and method improved based on YOLO v8. The system comprises a pod recognition network and a pod image restoration network. The pod recognition network adopts an improved YOLO v8 network combined with a hybrid attention module in a detection head to identify pods and stems, thereby obtaining a stem image in a pod image and an independent image of each pod in the pod image. The pod image restoration network adopts an adversarial neural network to restore the independent pod image. The present invention introduces an attention mechanism for extracting local and global features, so that a target frame is more accurately positioned in a target area, thereby improving detection accuracy. The method uses a loss function combined with ShapeIoU loss, calculates the loss by focusing on the shape and scale of the bounding box itself, thereby making the bounding box regression more accurate, and combines the calculation and statistics of the characteristics of the independent pod image to complete high-precision soybean seed testing.

Description

Technical Field

[0001] The present invention belongs to the technical field of soybean seed testing, and in particular relates to a soybean seed testing system and method improved based on YOLO v8. Background Art

[0002] Currently, there are two main types of high-throughput soybean bean and stem counting methods:

[0003] The first method involves threshing the soybean pods. The number of stem nodes is manually counted, and the beans are then spread out on a flat surface. Because the beans are all round, they avoid blocking each other and are distributed roughly evenly. A visible light camera is then used to capture the image, and digital image processing techniques are used to count the beans. This method takes a long time to thresh, resulting in smaller beans, and some beans may be damaged or lost, which can affect the accuracy of bean counting.

[0004] The second method involves removing pods from mature soybean plants, manually counting the number of stem nodes, and then neatly placing the pods on a flat plate. A visible light camera is used to image the pods, and digital image processing techniques are used to identify pods that are not obstructed by each other. Pods containing different types of beans are then manually classified, and the total number of beans is finally calculated. While this method avoids missed detections, it requires a lot of manual work, is time-consuming, and is not compatible with extracting obscured pods.

[0005] In addition, in actual situations, due to random placement, pods are easily blocked by each other, and even large areas of overlap may occur. At this time, the pods are classified into five categories, which belongs to multi-category detection. Then a large number of detection frames will be generated in this area, and the confidence of the category to which each detection frame belongs will be relatively large. Traditional suppression methods cannot eliminate erroneous candidate frames of different categories, so a target pod will generate candidate frames of multiple categories, resulting in a large number of errors and false detection problems.

[0006] As can be seen above, none of the aforementioned methods can quickly and accurately count soybean pods and stems. Direct counting results in large errors due to the small size of the pods, while indirect counting relies on manual placement and classification, which is time-consuming. Existing methods also suffer from limitations such as insufficient model generalization, low multi-scale object detection accuracy, and high data annotation costs. Summary of the Invention

[0007] In light of this, the present invention aims to provide an improved soybean seed identification system and method based on YOLO v8. This system uses the improved YOLO v8 network to identify pods and stems, and uses Shape_IOU and NMS to filter each detection frame to improve recognition accuracy. A repair network is then used to repair occluded pods, thereby resolving the issue of mutual occlusion between pods. By optimizing the network structure, enhancing data diversity, and refining training strategies, this invention significantly improves the efficiency and accuracy of phenotypic recognition in complex scenarios.

[0008] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0009] A soybean seed testing system based on an improved YOLO v8 includes a pod recognition network and a pod image restoration network. The soybean recognition network uses an improved YOLO v8 network, inputs a pod image into the YOLO v8 network for pod and stem recognition, and obtains detection frames for the selected pods and stems. Shape_IOU and NMS are used to filter each detection frame, and based on the filtered detection frames, a stem image in the pod image and an independent image of each pod in the pod image are obtained. A hybrid attention module is incorporated into the detection head of the YOLO v8 network, and the hybrid attention module performs local and global feature extraction on the input features in parallel. The pod image restoration network restores the independent pod image. The pod image restoration network includes a discriminator and a generator, wherein the generator is used to restore the input independent pod image, and the discriminator is used to determine the authenticity of the restored independent pod image.

[0010] Furthermore, the YOLO v8 network also includes a trunk module and a neck module; the trunk module extracts multi-scale features from the input pod image; the neck module fuses the multi-scale features and inputs the fused features into the detection head; the detection head uses a hybrid attention module to identify pods based on the fused features, and segment the stem image and the independent image of each pod from the pod image.

[0011] Furthermore, in the detection head, the hybrid attention module extracts local and global features from the input features, decouples the extracted features, and completes the pod category and position prediction;

[0012] The hybrid attention module includes a local branch and a global branch. In the local branch, a 1×1 convolution is performed on the input image to extract channel features, the extracted features are shuffled, and then a 3×3 convolution is performed on the shuffled features to obtain local features containing channel information. In the global branch, a cross-correlation attention operation is performed on the input image to obtain a key matrix, a key matrix, and a value matrix. After multiplying the corresponding elements of the key matrix and the key matrix, the multiplied features are processed by the softmax function to obtain a feature map containing spatial features. After multiplying the feature map with the corresponding elements of the value matrix, the multiplied features are added to the corresponding elements of the input features to obtain global features. The corresponding elements of the local features and the global features are added to obtain the output features.

[0013] Furthermore, in the process of screening each detection box using Shape_IOU and NMS:

[0014] Use Shape_IOU to calculate the IOU value between two adjacent detection frames; based on the IOU value, use NMS to filter out the detection frames; calculate the confidence of the filtered detection frames, and select the detection frame with the highest confidence as the candidate frame; use the center point of each detection frame as the reference to search the set pixel range: if other center points are found, compare the confidence of the current detection frame with the confidence of the candidate frame, retain the detection frame with a confidence greater than 50%, and delete the detection frame with a confidence less than 50%, until the last detection frame is screened.

[0015] Furthermore, the generator adopts a U-Net structure. In the discriminator, the generator's restored pod image and the corresponding real pod image are subjected to multiple continuous convolution operations to complete feature extraction. The extracted features are subjected to subsequent image segmentation operations in PatchGAN to complete feature comparison, and an authenticity score is given to evaluate the authenticity of the pod image. The authenticity score is used to guide the generator to complete the restoration of the pod image.

[0016] A soybean seed testing method improved based on YOLO v8, comprising:

[0017] S1: Obtain a pod image dataset and preprocess it to obtain a training set that includes pod occlusion conditions;

[0018] S2: Using the training set obtained in step S1, the soybean seed detection system improved based on YOLO v8 provided by the present invention is trained to obtain a soybean seed detection model;

[0019] S3: According to the training results of step S2, the hyperparameters of the soybean seed selection model are adjusted until the optimal soybean seed selection model is obtained;

[0020] S4: Input the pod image to be tested into the optimal soybean seed testing model obtained in step S4 to obtain a repaired independent pod image of each pod in the pod image, as well as a stem image; calculate and count the traits of the stem image and all independent pod images, thus completing the soybean seed testing.

[0021] Furthermore, step S1 includes:

[0022] Mature soybean plants were collected in different scenes. The soybean pods were randomly spread on the top of the background board, and the stems of the soybean plants were placed below the background board. The pod images were obtained by using a camera.

[0023] Remove the redundant parts of the pod image and crop the pod image into an image of uniform pixel size. Annotate the pods and stems in the cropped image to obtain an annotation file.

[0024] Image enhancement is performed on the cropped images and the annotation files to obtain a training set, which includes the cropped pod images and the corresponding annotation files.

[0025] Furthermore, in step S2, the soybean recognition network is trained using the recognition loss function, and the pod image restoration network is trained using the restoration objective function; wherein,

[0026] The recognition loss function is:

[0027] LOSS=W BCE ×LOSS BCE +LOSS ShapeIoU ×W ShapeIoU +LOSS DFL ×W DFL ;

[0028] Among them, LOSS represents the recognition loss function, LOSS BCE represents the binary cross entropy loss function, W BCE Represents the weight of the binary cross entropy loss function, LOSS ShapeIoU Represents the ShapeIoU loss function, W ShapeIoU Represents the weight of the ShapeIoU loss function, LOSS DFL represents the distribution focus loss function, W DFL represents the weight of the distribution focus loss function;

[0029] The repair objective function is:

[0030] G ’ =argminmaxLOSS cGAN (G,D)+λLOSS1(G);

[0031] Among them, G ’ represents the repair objective function, D represents the discriminator, G represents the generator, LOSS1 represents the L1 loss function, and λ represents the weight; cGAN (G,D) represents the repair loss function, which is:

[0032] LOSS cGAN (G,D)=E x,y [logD(x,y)]+E x,z [log(1-D(x,G(x,y)))];

[0033] Among them, x represents the real image, y represents the corresponding constraint, z represents the noise, E x,y [logD(x,y)] represents the discriminator’s expectation of the real image, E x,z [log(1-D(x,G(x,y)))] represents the discriminator’s expectation of the image restored by the generator.

[0034] Furthermore, the expression of the ShapeIoU loss function is as follows:

[0035] LOSS ShapeIoU =1-IoU+distance shape +0.5×Ω shape ;

[0036] Among them, 1-IoU represents the basic IoU loss, distance shape The distance metric representing the bounding box shape, Ω shape Represents the shape weight term.

[0037] Furthermore, in step S4, the pod traits include pod length, pod width, pod circumference and pod projected area; the pod trait extraction process includes: grayscale the repaired pod independent image, extracting the R channel image with the maximum contrast between the pod grains and the background; binarizing the R channel image using a threshold segmentation algorithm to obtain a pod binary image; extracting the pod outer contour based on the pod binary image, including the coordinates of each point on the contour; and calculating the pod length, width, circumference and pod projected area based on the obtained contour coordinates.

[0038] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0039] (1) In the soybean seed detection system based on the improved YOLO v8, the present invention introduces an attention mechanism into the detection head to further focus on the target features of different types of pods in the feature map, thereby quickly and accurately focusing on the target area, giving the target area feature map a higher weight, thereby highlighting the target and suppressing other irrelevant areas, allowing the target frame to be more accurately positioned in the target area, and improving detection accuracy; in addition, the present invention adopts a combination of Shape_IOU and NMS to retain the most accurate detection frames of different categories and remove redundant detection frames, thereby improving recognition accuracy and avoiding missed detection and false detection of targets;

[0040] (2) In the improved soybean seed detection method based on YOLO v8 described in the present invention, a loss function combined with ShapeIoU loss is used to solve the problem of being unable to eliminate erroneous candidate frames of different categories, which results in multiple categories of candidate frames being generated for one target pod, ultimately causing a large number of errors and false detections. The conditional constraints on the generation of the target frame during training are unreasonable. When detecting dense and overlapping areas of pod images, the shape and scale of the bounding box are more important. The original regression method only considers the geometric relationship between the target box and the prediction box, and uses the relative position and shape of the bounding box to calculate the loss, while ignoring the influence of inherent attributes such as the shape and scale of the bounding box on the bounding box regression, resulting in low accuracy when detecting dense and overlapping areas. The present invention uses ShapeIoU_Loss to focus on the shape and scale of the bounding box itself. This method can calculate the loss by focusing on the shape and scale of the bounding box itself, thereby making the bounding box regression more accurate. This solves the problem that the existing network uses the CIoU_Loss loss function, which only considers the overlapping area, center point distance, and aspect ratio between the target box and the generated box. This makes the target box regression process more stable, with faster convergence speed and higher convergence accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0042] Figure 1 This is a schematic diagram of the improved soybean seed testing system based on YOLO v8 according to an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of a pod identification network according to an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a hybrid attention module according to an embodiment of the present invention;

[0045] Figure 4A schematic diagram of a peapod image restoration network according to an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of a generator according to an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of a discriminator according to an embodiment of the present invention;

[0048] Figure 7 A schematic diagram of a soybean seed testing method improved based on YOLO v8 according to an embodiment of the present invention;

[0049] Figure 8 Creating the pod image described in the embodiment of the present invention;

[0050] Figure 9 This is a diagram of the pod target recognition result described in the embodiment of the present invention;

[0051] Figure 10 This is a partial enlarged view of the pod target recognition described in the embodiment of the present invention;

[0052] Figure 11 A partially enlarged view of the stem target identification according to an embodiment of the present invention;

[0053] Figure 12 A comparison chart of the results of the pod image restoration described in the embodiment of the present invention;

[0054] Figure 13 This is a diagram of the process of extracting pod traits according to an embodiment of the present invention;

[0055] Figure 14 The data fitting effect diagram of the bad pod number statistics described in the embodiment of the present invention is created

[0056] Figure 15 This is a data fitting effect diagram of the statistics of the number of pods of a type described in an embodiment of the present invention;

[0057] Figure 16 This is a data fitting effect diagram of the second type of pod number statistics described in the embodiment of the present invention;

[0058] Figure 17 This is a data fitting effect diagram of the statistics of the number of three types of pods described in the embodiment of the present invention;

[0059] Figure 18 This is a data fitting effect diagram of the statistics of the number of four types of pods described in the embodiment of the present invention;

[0060] Figure 19 This is a data fitting effect diagram of the statistics of the number of nodes in the stem according to the embodiment of the present invention;

[0061] Figure 20 This is a data fitting effect diagram of the statistics of the number of all pods described in the embodiment of the present invention;

[0062] Figure 21 This is a data fitting effect diagram for the statistics of the number of all beans described in the embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0064] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0065] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0066] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0067] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0068] like Figures 1 to 6As shown, the soybean seed testing system based on the improved YOLO v8 described in the embodiment of the present invention includes a pod recognition network and a pod image restoration network. Among them, the soybean recognition network adopts the improved YOLO v8 network, inputs the pod image into the YOLO v8 network to identify the pods and stems, and obtains the detection frame of the selected pods and stems; uses Shape_IOU and NMS to filter each detection frame, and obtains the stem image in the pod image and the independent image of each pod in the pod image based on the filtered detection frame. The detection head of the YOLO v8 network is combined with a hybrid attention module, which performs local and global feature extraction on the input features in parallel. The pod image restoration network repairs the independent pod image; the pod image restoration network includes a discriminator and a generator, the generator is used to repair the input independent pod image, and the discriminator is used to judge the authenticity of the restored independent pod image.

[0069] The YOLO v8 network also includes a trunk module and a neck module. The trunk module extracts multi-scale features from the input pod image. The neck module fuses these multi-scale features and inputs them into the detection head. The detection head uses a hybrid attention module to identify the pods based on the fused features and segment the pod image into a stem image and individual images of each pod. In this embodiment of the present invention, both the trunk module and the neck module adopt the trunk module and the neck module of the basic YOLO v8 network.

[0070] The backbone modules in the YOLO v8 network include the input layer, convolutional layer, pooling layer, activation layer, and output layer: the input layer is used to input the pod image captured by the imaging device, the convolutional layer is used to extract information from the image, the pooling layer is used to select the information extracted by the convolutional layer and reduce its dimensionality, the activation layer is used to improve the network's nonlinear fitting capabilities, and the output layer is used to output feature maps rich in high-level image semantic information after multiple convolutional and pooling activations. The core module in the YOLO v8 network adopts a pyramid structure, including convolutional layers, upsampling layers, and downsampling layers: the convolutional layer is used to reduce the dimensionality of the feature maps output at each stage, the upsampling layer is used to upsample the feature maps obtained by the network, and the downsampling layer transmits the upsampled semantic features to enhance the semantic information of the entire pyramid. The detection head in the YOLO v8 network includes a classification channel, a regression channel, and candidate box suppression. The classification channel consists of several convolutional layers and is used to classify the categories of the generated candidate boxes. The regression channel consists of several convolutional layers and generates candidate box parameters through anchor-free generation, including the coordinates of the top and bottom right points of the target box. Candidate box suppression is used to eliminate redundant detection boxes and output the final target detection results.

[0071] Since there are a large number of pod targets that need to be identified and measured, the pod detection is performed through random placement, resulting in uneven distribution of pods, and there are many types of pods, and the distinction between various pod types is not obvious on certain specific varieties of soybeans. The above problems make it difficult to extract soybean target features, and it is impossible to focus on the target area quickly and accurately, and to distinguish the target results, resulting in low target recognition accuracy and failure to meet the recognition task requirements. Therefore, the present invention introduces a hybrid attention module that performs local and global feature extraction in the detection head to improve the recognition accuracy and robustness of the system. Specifically, in some embodiments, the hybrid attention module performs local and global feature extraction on the input features, decouples the extracted features, and completes the category prediction and position prediction of the pods. In an embodiment of the present invention, the hybrid attention module is specifically placed after the three output layers of the neck module. The hybrid attention module consists of a local branch and a global branch. The local branch aims to extract local features from the feature map through convolution and channel shuffling. The local branch focuses on extracting local information from the pod image to facilitate the comprehensive modeling of global and local features in the feature map. The global branch utilizes an attention mechanism to model long-range feature dependencies. This attention mechanism enables the system to capture data information from a wider range of soybean targets, thereby better understanding the global characteristics of the feature map. In the local branch, a 1×1 convolution is performed on the input image to extract channel features, and the extracted features are then subjected to channel shuffling. In this embodiment of the present invention, a 3×3 convolution is also performed on the channel shuffled features to obtain local features. In the global branch, a cross-correlation attention operation is performed on the input image to obtain a key matrix, a key matrix, and a value matrix. In this embodiment of the present invention, the global branch performs a 1×1 convolution on the input image, and then performs three 3×3 depthwise separable convolutions on the convolved features to obtain a key matrix Q, a key matrix K, and a value matrix V. After multiplying the corresponding elements of the key matrix and the key matrix, the multiplied features are processed by the softmax function to obtain a feature map containing spatial features; after multiplying the feature map with the corresponding elements of the value matrix, the multiplied features are added to the corresponding elements of the input features to obtain global features. In an embodiment of the present invention, a 1×1 convolution operation is also performed on the value matrix, and the features after the 1×1 convolution operation are multiplied with the corresponding elements of the feature map containing spatial features. Adding the corresponding elements of the local features and the global features can effectively combine local and global information, improve the model's ability to understand the pod image, and obtain output features that are more focused on the target area. The present invention further focuses on the target features of different types of pods in the feature map by setting a hybrid attention module for local and global feature extraction in the detection head, thereby quickly and accurately focusing on the target area, giving the target area feature map a higher weight, thereby highlighting the target and suppressing other irrelevant areas, allowing the target frame to be more accurately positioned in the target area, and improving detection accuracy.

[0072] The present invention provides a candidate box suppression method to replace the classic NMS (non-maximum suppression) to retain the most accurate detection box and remove redundant detection boxes. Specifically, in some embodiments, in the process of using Shape_IOU and NMS to screen each detection box: Shape_IOU is used to calculate the IOU value between two adjacent detection boxes; based on the IOU value, NMS is used to filter the detection boxes; the confidence of the filtered detection boxes is calculated, and the detection box with the highest confidence is selected as the candidate box; with the center point of each detection box as the reference, a set pixel range is searched: if other center points are found, the confidence of the current detection box is compared with the confidence of the candidate box, the detection box with a confidence greater than 50% is retained, and the detection box with a confidence less than 50% is deleted, until the last detection box is screened.

[0073] In an embodiment of the present invention, Shape_IOU is used to calculate the IOU value between each detection frame, and the NMS calculation is combined to obtain the candidate frame with the highest confidence in the same category and exceeding the set IOU threshold. Due to dense and large-scale occlusion, a large number of candidate frames of other categories may appear in an area, resulting in a large number of false detection operations. At this time, the detection frame suppressed by traditional NMS and the confidence of the corresponding detection frame are read, and the detection frame coordinates are converted into center point coordinates. The set pixel range is searched based on the center point of each target frame. If other center points are found, their confidence is compared with the confidence of the candidate frame, and the detection frame with the highest confidence is retained, and the detection frame with lower confidence is deleted. This process is repeated until the last candidate frame.

[0074] In some embodiments, the pod image restoration network adopts a PIX2PIX network model. Specifically, the generator is an existing U-Net structure. In the discriminator provided in some embodiments, multiple convolutions are used to replace the image blocking operation in PatchGAN, that is, the pod independent image repaired by the generator and the corresponding real pod independent image are subjected to multiple consecutive convolution operations to complete feature extraction, and the extracted features are subjected to subsequent image blocking operations in PatchGAN to complete feature comparison, and an authenticity score is given to evaluate the authenticity of the pod independent image, and the authenticity score is used to guide the generator to complete the restoration of the pod independent image. In an embodiment of the present invention, after performing four consecutive convolution operations on the input image, feature extraction is completed, and the extracted features are input into the PatchGAN module.

[0075] A soybean seed testing method based on YOLO v8, combined with Figure 7 , methods include:

[0076] S1: Obtain a pod image dataset and preprocess it to obtain a training set that includes pod occlusion conditions.

[0077] In some embodiments, step S1 includes:

[0078] S11: Capturing mature soybean plants in different scenarios, randomly spreading the soybean pods on the plants above a background plate, placing the soybean stems below the background plate, and photographing them using a camera to obtain images of the pods. In this embodiment of the present invention, the scenarios for capturing mature soybean plants include natural growth scenarios, adverse stress scenarios (such as drought, salinization, waterlogging, etc.), and biological stress scenarios (such as insect pests, bacteria, viruses, etc.), and photographing the pods and stems on the background plate using a camera.

[0079] like Figure 8 As shown, there are many different pod occlusion scenarios, with significant randomness, including but not limited to: no occlusion or only slight interference; occlusion of the top, middle, tail, or sides of a single pod; and occlusion of multiple pods side by side or staggered. This makes dataset creation challenging. Insufficient occlusion modeling in a dataset can lead to unknown network repair scenarios, resulting in repair failures and, indirectly, the inability to extract phenotypic traits from the pods. Therefore, embodiments of the present invention consider all possible occlusion scenarios and image each type of occlusion separately.

[0080] S12: Redundant portions of the pod image are removed, and the pod image is cropped to a uniform pixel size. The pods and stems in the cropped image are labeled to generate a labeling file. In this embodiment of the present invention, the collected pod image is labeled with the corresponding pod and stem type labels using the LabelImg tool. Bean pod types include bad pods, first-class pods, second-class pods, third-class pods, and fourth-class pods.

[0081] S13: Perform image enhancement on the cropped image and the annotation file to obtain a training set, which includes the cropped bean pod image and the corresponding annotation file. Image enhancement includes HSV color gamut adjustment, random image scaling, random image cropping, and mosaic operation of partial image areas on the cropped image and the annotation file. In an embodiment of the present invention, image enhancement is used to simulate the same occlusion situation at different angles, HSV color gamut adjustment is used to simulate soybean images imaged in different environments, including adjusting the brightness, contrast, saturation, and hue parameters of the soybean image for simulation, and mosaic operation of partial image areas is used to simulate the imaging device focusing failure resulting in blurred imaging. The present invention can simulate the pod occlusion situation in various situations by performing data enhancement on the original bean pod image, and greatly expands the data set. Each enhancement method gives the pod image repair network new data, thereby improving the robustness of the repair network.

[0082] S2: Using the training set obtained in step S1, the improved soybean seed detection system based on YOLO v8 provided by the present invention is trained to obtain a soybean seed detection model. Specifically, in this embodiment of the present invention, a pod recognition network is trained using pod images in the training set as input and annotated files in the training set as output. A pod image restoration network is trained using recognized individual pod images as input and corresponding complete, real individual pod images as output.

[0083] In some embodiments, the soybean recognition network is trained using the recognition loss function, and the pod image restoration network is trained using the restoration objective function. During the training process, unreasonable constraints on the generation of the target box may occur, that is, when detecting dense and overlapping areas of the pod image, the shape and scale of the bounding box are more important. In the original regression method, only the geometric relationship between the target box and the prediction box is considered, and the loss is calculated using the relative position and shape of the bounding box, while ignoring the influence of inherent attributes such as the shape and scale of the bounding box on the bounding box regression, resulting in low accuracy when detecting dense and overlapping areas. Based on this, the present invention uses a bounding box regression method that uses ShapeIoU loss to focus on the shape and scale of the bounding box itself. This method can calculate the loss by focusing on the shape and scale of the bounding box itself, thereby making the bounding box regression more accurate. It solves the deficiency of the CIoU_Loss loss function that only considers the overlapping area, center point distance and aspect ratio between the target box and the generated box, making the target box regression process more stable, and the convergence speed is faster and the convergence accuracy is higher. Specifically: the recognition loss function is:

[0084] LOSS=W BCE ×LOSS BCE +LOSS ShapeIoU ×W ShapeIoU +LOSS DFL ×W DFL ;

[0085] Among them, LOSS represents the recognition loss function, LOSS BCE represents the binary cross entropy loss function, W BCE Represents the weight of the binary cross entropy loss function, LOSS ShapeIoU Represents the ShapeIoU loss function, W ShapeIoU Represents the weight of the ShapeIoU loss function, LOSS DFL represents the distribution focus loss function, W DFL Represents the weight of the distribution focus loss function. Weight W BCE 、W ShapeIoU and W DFL According to the actual situation, adaptive adjustment is made. In the embodiment of the present invention, W BCE =0.5, W ShapeIoU= 0.75, and W DFL =1.5.

[0086] Binary cross entropy loss function LOSS BCE for:

[0087] ;

[0088] Among them, N represents the number of groups of model prediction objects, p(y i ) represents the probability of the output label, y i Represents the true binary label corresponding to the i-th predicted object;

[0089] The expression of ShapeIoU loss function is as follows:

[0090] LOSS ShapeIoU =1-IoU+distance shape +0.5×Ω shape ;

[0091] Among them, 1-IoU represents the basic IoU (intersection over union) loss, specifically:

[0092] ;

[0093] Among them, B represents the prediction box of the recognition target, B gt Indicates the real box of the recognized target; distance shape Distance metrics representing the shape of the bounding box, specifically:

[0094] ;

[0095] Among them, (x c ,y c ) represents the center position of the prediction box, Represents the center position of the real box, c represents the diagonal distance between the predicted box and the real box in the minimum closed bounding box, hh represents the weight coefficient in the vertical direction, and ww represents the weight coefficient in the horizontal direction. Specifically:

[0096] ;

[0097] ;

[0098] Among them, w gt and h gt Represent the width and height of the real box respectively, scale represents the scaling factor, which is related to the size of the target in the dataset; Ω shape Represents the shape weight term, specifically:

[0099] ;

[0100] ;

[0101] ;

[0102] Where θ=4.

[0103] Distribution Focus Loss Function for:

[0104] .

[0105] The repair objective function is:

[0106] G'=argminmaxLOSS cGAN (G,D)+λLOSS1(G);

[0107] Among them, G' represents the restoration objective function, D represents the discriminator, G represents the generator, LOSS1 represents the L1 loss function, and λ represents the weight; LOSS cGAN (G,D) represents the repair loss function, which is:

[0108] LOSS cGAN (G,D)=E x,y [logD(x,y)]+E x,z [log(1-D(x,G(x,y)))];

[0109] Among them, x represents the real image, y represents the corresponding constraint, z represents the noise, E x,y [logD(x,y)] represents the discriminator’s expectation of the real image, E x,z [log(1-D(x,G(x,y)))] represents the discriminator’s expectation of the image restored by the generator.

[0110] S3: According to the training results of step S2, the hyperparameters of the soybean seed selection model are adjusted until the optimal soybean seed selection model is obtained. In the embodiment of the present invention, the hyperparameters of the training include:

[0111] The hyperparameters for training the pod recognition network include: image size of 640×640, batch size of 16, number of classes of 6, initial learning rate of 1e-3, annealed cosine training strategy, final learning rate reduced to 1e-6, Adam optimizer, and 400 iterations.

[0112] The hyperparameters for training the BeanPod image inpainting network include: image size of 256×256 during training, batch size of 16, initial learning rate of 2e-4, annealed cosine training strategy, final learning rate reduced to 1e-6, Adam optimizer, and 200 iterations.

[0113] S4: Input the pod image to be tested into the optimal soybean seed testing model obtained in step S4 to obtain a repaired independent pod image of each pod in the pod image, as well as a stem image; calculate and count the traits of the stem image and all independent pod images, thus completing the soybean seed testing.

[0114] The pod identification result in the embodiment of the present invention is as follows Figures 9 to 11 As shown, the pod repair effect in the embodiment of the present invention is as follows Figure 12 shown. Figure 9 and Figure 10 The bounding boxes of different colors represent the different types of pods that have been identified. The types of pods identified and their class probabilities are marked above the bounding boxes. Figure 10 Among them, soybeon_1 represents the first type of pod, soybeon_2 represents the second type of pod, soybeon_3 represents the third type of pod, and soybeon_4 represents the fourth type of pod ( Figure 10 not shown), with Figure 10 For example, the label "soybeon_1 0.84" on one of the bounding boxes indicates that the pod in the bounding box is identified as a type 1 pod, and the probability of it being a type 1 pod is 0.84, or 84%. Figure 9 and Figure 11 The green box in the image represents the identified stem and its probability of being a stem. Figure 11 For example, if one of the bounding boxes is labeled "stem 0.61", it means that the content in the bounding box is a stem, and the probability that it is a stem is 0.61, or 61%. Figure 12 The middle left column shows different degrees of occlusion (no occlusion with slight interference, side occlusion, and side-by-side occlusion of multiple pods). Figure 12 The right column shows the corresponding repaired results. Figure 12 It can be seen from the figure that the present invention can effectively restore pods with different degrees of occlusion, laying a good foundation for subsequent pod trait statistics.

[0115] In some embodiments, the pod traits in step S4 include pod length, pod width, pod circumference and pod projected area. Figure 13 As shown, it includes: graying the restored pod independent image, extracting the grayscale image of the R channel with the largest contrast between the pod grains and the background for subsequent shape extraction, as shown in FIG. Figure 13(a) in the figure; use the threshold segmentation algorithm to binarize the R channel image and obtain the pod binary image, as shown in Figure 13 (b) in the figure; extract the outer contour of the pod based on the binary image of the pod, including the coordinates of each point on the contour; calculate the length, width, perimeter and projection area of ​​the pod based on the obtained contour coordinates, as shown in Figure 13 (c) in the.

[0116] In an embodiment of the present invention, in the process of obtaining the length of the pod, two points on the outer contour of the pod are arbitrarily selected, and the distance between the two points is calculated. The maximum value of all distances is the length of the pod. In the process of obtaining the width of the pod, the normal slope of the straight line between the two points with the maximum distance is calculated, and then the distance between the intersection of the normal with the same normal slope and the outer contour of the pod under different intercepts is calculated. The maximum value of all distances is the width of the pod. In the process of obtaining the perimeter of the pod, the sum of the distances between all adjacent pixel points on the outer contour of the pod is the perimeter of the pod. In an embodiment of the present invention, based on the points on the contour of the pod, if the two points are adjacent in the upper and lower or left and right directions, the distance between the two points is defined as 1. If the two points are adjacent in the upper left, lower left, upper right or lower right directions, the distance between the two points is defined as , the sum of the distances between all adjacent pixels on the contour is the pod perimeter. The pod projected area is the number of pixels within the pod's outer contour.

[0117] In an embodiment of the present invention, the stem traits include the number of main stem nodes, the maximum internode distance, the minimum internode distance, the average internode distance, the total length of the stem nodes, the farthest internode distance, and the curvature. Specifically, the target frame parameters corresponding to the label value are read according to the return value obtained by identification, including the coordinates (x1, y1, x2, y2) of the original image corresponding to the upper left point and the lower right point of the target frame, and the length and width of the stem are obtained as x2-x1 and y2-y1 respectively, and the coordinate position of the original image where the center point of the stem is located is ((x2-x1) / 2+x1, (y2-y1) / 2+y1). According to the obtained center coordinates of the first and last stems, the approximate length of the main stem can be obtained; according to the center coordinates of the adjacent stems, the lengths of several main trunks can be obtained. Count the distances between adjacent stems (X1, X2, ..., X n ), and obtain the maximum value X by comparison MAX , X MAX is the maximum internode distance. Similarly, the minimum value X is obtained by comparison. MIN , X MIN The minimum internode distance is obtained by averaging the distances between adjacent stems (X1, X2, ..., X n ) the average value X MEAN , X MEAN is the average internode distance, and the total length of the stem node X is obtained by accumulation. SUM =X1+X2+…+X n; Get the center coordinates of the first stem node (X1, Y1) and the center coordinates of the last stem node (X n ,Y n ), through the formula The ratio of the farthest internode distance to the total length of the stem node is the curvature, which is used to calculate the degree of curvature of the stem.

[0118] In an embodiment of the present invention, the number of pods contained in different tags can also be read from the number of beans in the entire plant, multiplied by the number of pods determined by the tags, and then added together to obtain the number of beans in the entire plant. For example, a bad pod is defined as P0, which contains no pods, that is, the number of pods corresponding to the bad pod is 0; a first-class pod is defined as P1, which contains 1 pod, that is, the number of pods corresponding to the first-class pod is 1; a second-class pod is defined as P2, which contains 2 pods, that is, the number of pods corresponding to the second-class pod is 2; a third-class pod is defined as P3, which contains 3 pods, that is, the number of pods corresponding to the third-class pod is 2; a fourth-class pod is defined as P4, which contains 3 pods, that is, the number of pods corresponding to the fourth-class pod is 2. In this case, the number of beans in the entire plant = 0×P0+1×P2+2×P2+3×P3+4×P4.

[0119] The statistical data fitting effects of the number of bad pods, first-class pods, second-class pods, third-class pods, fourth-class pods and stem nodes in the embodiment of the present invention correspond to Figures 14 to 19 The statistical data fitting effects of the number of all pods and the number of all beans correspond to Figure 20 and Figure 21 . Figures 14 to 21 The vertical axis represents the number of corresponding pods or stem nodes counted using the method provided by the present invention, and the horizontal axis represents the number of corresponding pods or stem nodes counted manually. Figures 14 to 21 It can be seen that the number of pods or stem nodes counted by the method provided by the present invention is close to the corresponding number counted manually, indicating that the method provided by the present invention can effectively complete the counting of the number of pods or stem nodes. 2 The R-squared indicator reflects the accuracy of quantitative statistics, where R 2 Also known as the coefficient of determination or determination coefficient, it is an indicator used to measure the goodness of fit of the regression model to the observed data. It indicates the degree to which the independent variable in the model explains the dependent variable, and its value range is between 0 and 1. 2 The closer it is to 1, the better the model fits the data; the closer it is to 0, the weaker the model's ability to explain the data. The R values ​​of the number of bad pods, first-class pods, second-class pods, third-class pods, fourth-class pods, stem nodes, all pods, and all beans counted using the method provided by the present invention are as follows: 2The indicators are 0.9827, 0.9638, 0.9914, 0.9904, 0.9352, 0.7031, 0.9896 and 0.9941 respectively. It can be seen that R 2 The indices are all close to 1, which further illustrates that the fitting effect of the method provided by the present invention is very good, that is, there is a highly significant linear relationship between the method provided by the present invention and the manual recognition method.

[0120] In addition, the embodiment of the present invention uses five indicators, namely precision P (Precision), recall R (Recall), mAP_50, mAP_75, and mAP50-95, to evaluate the target detection performance of the soybean recognition network. Among them, precision P refers to the proportion of samples predicted as positive that are actually positive, which measures the accuracy of the prediction results; recall R is the proportion of true positive samples that are correctly predicted as positive, reflecting the model's ability to capture positive samples; mAP_50 represents the average precision when the IoU threshold is 0.5, reflecting the detection performance of the model on different categories; mAP_75 represents the average precision when the IoU threshold is 0.75, which has higher detection accuracy requirements than mAP_50; mAP50-95 represents the average precision in the IoU threshold range of 0.5 to 0.95, which comprehensively measures the performance of the model at different IoU thresholds. The indicator evaluation results are shown in Table 1:

[0121] Table 1: Index evaluation results

[0122]

[0123] As shown in Table 1, the overall performance of the method is as follows: precision (P) is 0.873, and recall (R) is 0.927, indicating that the method provided by the present invention has good accuracy in detecting various targets and capturing positive samples. Among the mAP metrics, mAP50 is 0.925, indicating good detection performance at an IoU of 0.5; mAP95 is 0.101, indicating relatively weak performance under the strict IoU of 0.95; and mAP50-95 is 0.716, indicating that the overall performance of the method provided by the present invention is above average. When analyzing soybean type subdivisions (class I, class II, class III, and class IV), these subdivisions generally perform well across all metrics. For example, when detecting class III, the precision (P) is 0.955, the recall (R) is 0.966, and the mAP50-95 reaches 0.846, demonstrating that the method provided by the present invention has high detection accuracy and overall performance for these subdivisions.

[0124] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0125] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A soybean seed detection system based on YOLO v8, characterized in that: It includes pod recognition network and pod image restoration network; among them, The pod recognition network adopts an improved YOLO v8 network, inputs the pod image into the YOLO v8 network to identify the pods and stems, and obtains the detection frames of the selected pods and stems; uses Shape_IOU and NMS to filter each detection frame, and obtains the stem image in the pod image and the independent image of each pod in the pod image based on the filtered detection frame; the detection head of the YOLO v8 network is combined with a hybrid attention module, which performs local and global feature extraction on the input features in parallel; the YOLO v8 network also includes a trunk module and a neck module; the trunk module extracts multi-scale features from the input pod image; the neck module fuses the multi-scale features and inputs the fused features into the detection head; the detection head uses the hybrid attention module to recognize the pods based on the fused features, and segments the stem image and the independent image of each pod from the pod image; In the detection head, the hybrid attention module extracts local and global features from the input features, decouples the extracted features, and completes the category prediction and position prediction of the pod; the hybrid attention module includes a local branch and a global branch; In the process of using Shape_IOU and NMS to screen each detection frame: use Shape_IOU to calculate the IOU value between two adjacent detection frames; based on the IOU value, use NMS to filter the detection frames; calculate the confidence of the filtered detection frames, and select the detection frame with the highest confidence as the candidate frame; use the center point of each detection frame as the reference to search the set pixel range: if other center points are found, compare the confidence of the current detection frame with the confidence of the candidate frame, retain the detection frame with a confidence greater than 50%, and delete the detection frame with a confidence less than 50%, until the last detection frame is screened; The pod image restoration network restores the independent pod image; the pod image restoration network includes a discriminator and a generator, and the generator is a U-Net structure; in the discriminator, the independent pod image restored by the generator and the corresponding real independent pod image are subjected to multiple continuous convolution operations to complete feature extraction; the extracted features are subjected to subsequent image block operations in PatchGAN to complete feature comparison, and an authenticity score is given to evaluate the authenticity of the independent pod image, and the authenticity score is used to guide the generator to complete the restoration of the independent pod image.

2. The soybean seed detection system based on YOLO v8 according to claim 1, characterized in that: In the local branch, a 1×1 convolution is performed on the input image to complete channel feature extraction, the extracted features are channel shuffled, and then a 3×3 convolution is performed on the shuffled features to obtain local features containing channel information; In the global branch, a cross-correlation attention operation is performed on the input image to obtain a key matrix, a key matrix, and a value matrix; after multiplying the corresponding elements of the key matrix and the key matrix, the multiplied features are processed by a softmax function to obtain a feature map containing spatial features; after multiplying the feature map with the corresponding elements of the value matrix, the multiplied features are added to the corresponding elements of the input features to obtain a global feature; The local features and the corresponding elements of the global features are added together to obtain the output features.

3. A soybean seed detection method based on YOLO v8, characterized in that: include: S1: Obtain a pod image dataset and preprocess the pod image dataset to obtain a training set containing pod occlusion conditions; S2: Using the training set obtained in step S1, training the soybean seed detection system improved based on YOLO v8 as described in any one of claims 1 to 2 to obtain a soybean seed detection model; S3: According to the training result of step S2, adjusting the hyperparameters when training the soybean seed selection model until an optimal soybean seed selection model is obtained; S4: Input the pod image to be tested into the optimal soybean seed testing model obtained in step S4 to obtain a restored independent pod image of each pod in the pod image, as well as a stem image; calculate and count the traits of the stem image and all independent pod images, thereby completing the soybean seed testing.

4. The soybean seed testing method based on the improved YOLO v8 according to claim 3, characterized in that: Step S1 includes: Mature soybean plants are collected in different scenes, the pods on the soybean plants are randomly spread above a background plate, the stems of the soybean plants are placed below the background plate, and photographed using a camera to obtain pod images; removing redundant parts from the bean pod image, cropping the bean pod image into an image of uniform pixel size, and annotating the bean pods and stems in the cropped image to obtain an annotation file; Image enhancement is performed on the cropped images and the annotation files to obtain a training set, which includes the cropped pod images and the corresponding annotation files.

5. The soybean seed testing method based on the improved YOLO v8 according to claim 4, characterized in that: In step S2, the pod recognition network is trained using the recognition loss function, and the pod image restoration network is trained using the restoration objective function; wherein, The recognition loss function is: LOSS=W BCE ×LOSS BCE +LOSS ShapeIoU ×W ShapeIoU +LOSS DFL ×W DFL ; Among them, LOSS represents the recognition loss function, LOSS BCE represents the binary cross entropy loss function, W BCE Represents the weight of the binary cross entropy loss function, LOSS ShapeIoU Represents the ShapeIoU loss function, W ShapeIoU Represents the weight of the ShapeIoU loss function, LOSS DFL represents the distribution focus loss function, W DFL represents the weight of the distribution focal loss function; The repair objective function is: <h2 style=";text-align:left;direction:ltr">G'=argminmaxLOSS<h2 style=";text-align:left;direction:ltr"> cGAN <h2 style=";text-align:left;direction:ltr"> (G,D)+λLOSS1(G); Wherein, G' represents the restoration objective function, D represents the discriminator, G represents the generator, LOSS1 represents the L1 loss function, and λ represents the weight; LOSS cGAN (G,D) represents the repair loss function, which is: LOSS cGAN (G,D)=E x,y [logD(x,y)]+E x,z [log(1-D(x,G(x,y)))]; Among them, x represents the real image, y represents the corresponding constraint, z represents the noise, E x,y [logD(x,y)] represents the discriminator’s expectation of the real image, E x,z [log(1-D(x,G(x,y)))] represents the discriminator’s expectation of the image restored by the generator.

6. The soybean seed testing method based on the improved YOLO v8 according to claim 5, characterized in that: The expression of the ShapeIoU loss function is as follows: LOSS ShapeIoU =1-IoU+distance shape +0.5×Ω shape ; Among them, 1-IoU represents the basic IoU loss, distance shape The distance metric representing the bounding box shape, Ω shape Represents the shape weight term.

7. The soybean seed testing method based on the improved YOLO v8 according to claim 3, characterized in that: In step S4, the pod traits include pod length, pod width, pod circumference and pod projected area; the pod trait extraction process includes: graying the repaired pod independent image, extracting the R channel image with the maximum contrast between the pod grains and the background; binarizing the R channel image using a threshold segmentation algorithm to obtain a pod binary image; extracting the pod outer contour based on the pod binary image, including the coordinates of each point on the contour; and calculating the pod length, width, circumference and pod projected area based on the obtained contour coordinates.

Images