Repair system and method for seed pod shading and pod trait extraction method

Abstract

The present invention relates to the field of soybean seed testing technology, and in particular to a repair system and method for pod occlusion during seed testing, as well as a pod trait extraction method. The system includes a pod repair generator with an improved U-Net structure and a pod repair discriminator based on PatchGAN. The pod repair generator processes defective input pod images to obtain corresponding pod repair images. The present invention provides a multi-level residual convolution module and a self-attention module in the pod repair generator to perform multi-scale feature extraction and feature fusion to improve the pod image repair effect; the pod repair discriminator calculates the difference between the complete image corresponding to the input pod image and the corresponding pod repair image. The present invention repairs the pod image based on the repair system and completes the acquisition of trait information for each pod through a trait extraction method, thereby improving the statistical accuracy and efficiency of soybean pod phenotypic traits and avoiding the subjective errors caused by traditional manual statistics.

Description

Technical Field

[0001] The present invention belongs to the technical field of soybean seed testing, and in particular relates to a system and method for repairing soybean pod shading during seed testing, and a method for extracting soybean pod traits. Background Art

[0002] When detecting and classifying pod characteristics to indirectly count pods, to reduce manual labor and improve efficiency, it's often impossible to ensure even distribution of pods scattered on a flat surface. Consequently, pods will inevitably occlude each other, resulting in multiple pods within a detected pod target frame, and varying degrees of occlusion between pods. Therefore, it's difficult to accurately extract the characteristics of each pod based solely on the pod target frame output directly from the detection network.

[0003] In summary, how to design a method that can solve the problem of mutual occlusion between pods and provide a good premise for accurately extracting the characteristics of each pod is an urgent problem that needs to be solved. Summary of the Invention

[0004] In view of this, the present invention aims to provide a repair system and method for soybean pod occlusion and a method for extracting soybean pod traits, which uses an improved U-Net network with an attention mechanism and a residual convolution module to repair occluded or missing pod images, and then uses a PatchGAN-based pod repair discriminator to discriminate the repaired pod images. Finally, the trait information of each pod is obtained through a trait extraction method, thereby improving the statistical accuracy and efficiency of soybean pod phenotypic traits and avoiding the subjective errors caused by traditional manual statistics.

[0005] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0006] A repair system for detecting pod occlusion includes: a pod repair generator with an improved U-Net structure, which processes a defective input pod image to obtain a corresponding pod repair image; a multi-level residual convolution module is provided in the pod repair generator to perform multi-level feature extraction and enhancement on the input pod image; a pod repair discriminator based on PatchGAN calculates the difference between the complete image corresponding to the input pod image and the corresponding pod repair image; the pod repair generator includes a pod feature encoding branch and a pod feature decoding branch, and A self-attention module is set between the peapod feature decoding branch and the peapod feature encoding branch; in the peapod feature encoding branch, the input peapod image is continuously processed by multi-level residual convolution modules to obtain peapod encoding features, and the features output by each level of residual convolution module are jump-connected to the peapod feature decoding branch; in the self-attention module, the peapod encoding features are continuously subjected to self-attention operations and spatial and channel information enhancement to obtain peapod enhancement features; in the peapod feature decoding branch, the peapod enhancement features are step-by-step upsampled and decoded in combination with the features output by the peapod feature encoding branch to obtain a peapod repaired image.

[0007] Furthermore, each level of residual convolution module includes a first branch and a second branch; in the first branch, after performing multiple convolution operations on the input image or input features, the features after each convolution are spliced ​​to obtain spliced ​​features; in the second branch, after performing no less than one convolution operation on the input image or input features, the convolution features are added to the corresponding elements of the spliced ​​features, and the added features are subjected to SPD-Conv operation to complete lossless downsampling to obtain output features.

[0008] Furthermore, in the process of the self-attention module continuously performing self-attention operations on the peapod encoding features: after the peapod encoding features are upgraded in the channel dimension, the upgraded features are divided into three parts through the dimension slicing operation to obtain the query matrix, key matrix and value matrix; the key matrix of the query matrix is ​​multiplied and the probability matrix is ​​generated through the SoftMax operation, and after multiplying the probability matrix with the value matrix, the dimension is transformed and convolved, and then the result is added to the peapod encoding features to obtain the attention feature; in the process of the self-attention module strengthening the spatial and channel information: the attention feature is divided into two parts after the dimension is upgraded, the two features are respectively subjected to depthwise separable convolution and multiplied, the multiplied feature is convolved and then added to the attention feature to obtain the peapod enhancement feature.

[0009] Furthermore, the peapod feature decoding branch includes a cascade of multiple transposed convolutional layers, and the number of transposed convolutional layers is consistent with the number of residual convolutional modules; the first-level transposed convolutional layer combines the peapod enhancement features to upsample the peapod encoding features; each subsequent level of transposed convolutional layer combines the features of the corresponding scale output by the peapod feature encoding branch to upsample the features upsampled by the previous level of transposed convolutional layer, until the last level of transposed convolutional layer outputs the peapod repaired image.

[0010] Furthermore, in the pod repair discriminator, multiple continuous convolution operations are performed on the pod repair image and the corresponding real pod complete image to complete feature extraction. The extracted features are then subjected to subsequent image segmentation operations in PatchGAN to complete feature comparison, and a authenticity score is given to evaluate the pod repair image. The authenticity score is then used to guide the pod repair generator to complete the repair of the input pod image.

[0011] A method for repairing seed pod occlusion, comprising:

[0012] Obtain a pod image dataset and preprocess it to obtain a training set that includes pod occlusion conditions.

[0013] Using the training set, the repair system for detecting pod occlusion provided by the present invention is trained to obtain a pod repair model;

[0014] Based on the training results, adjust the hyperparameters when training the pod repair model until the optimal pod repair model is obtained;

[0015] The pod image to be repaired is input into the pod repair generator in the optimal pod repair model to obtain the corresponding pod repair image.

[0016] Furthermore, during the training of the repair system, the repair objective function is used to train the pod repair model;

[0017] The repair objective function is:

[0018] G'=argminmaxLOSS cGAN (G,D)+λ'LOSS1(G);

[0019] Among them, G' represents the repair objective function, D represents the pod repair discriminator, G represents the pod repair generator, LOSS1 represents the L1 loss function, and λ' represents the repair weight; LOSS cGAN (G,D) represents the repair loss function, which is:

[0020] LOSS cGAN (G,D)=E x,y [logD(x,y)]+E x,z[log(1-D(x,G(x,y)))];

[0021] Among them, x represents the real image, y represents the corresponding constraint, z represents the noise, E x,y [logD(x,y)] represents the expectation of the pod repair discriminator for the real image, E x,z [log(1-D(x,G(x,y)))] represents the expectation of the peapod inpainting discriminator for the image inpainted by the peapod inpainting generator.

[0022] A method for extracting bean pod traits comprises: obtaining a repaired bean pod image using the repair method for detecting bean pod occlusion provided by the present invention; gray-scaling the repaired bean pod image and extracting a grayscale image of a channel with the maximum contrast between the pod and the background; binarizing the grayscale image using a threshold segmentation algorithm to obtain a binary image of the pod; extracting the outer contour of the pod based on the binary image of the pod, and obtaining the bean pod traits based on the outer contour of the pod.

[0023] Furthermore, the pod traits include pod length, pod width, pod circumference and pod projected area.

[0024] Furthermore, in the process of obtaining the pod length, two points on the outer contour of the pod are arbitrarily selected, and the distance between the two points is calculated. The maximum value of all distances is the pod length; in the process of obtaining the pod width, the normal slope of the straight line between the two points with the maximum distance is calculated, and then the distance between the intersection of the normal with the same normal slope and the outer contour of the pod at different intercepts is calculated. The maximum value of all distances is the pod width; in the process of obtaining the pod perimeter, the sum of the distances between all adjacent pixel points on the outer contour of the pod is the pod perimeter; the pod projection area is the number of all pixels within the outer contour of the pod.

[0025] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0026] (1) The present invention creates a repair system, method, and pod trait extraction method for detecting pod occlusion. An improved U-Net network combined with an attention mechanism and a residual convolution module is used to perform multi-scale feature extraction and feature fusion on occluded or missing images, thereby improving the repair effect of the pod image. The repaired pod image is then discriminated by a pod repair discriminator based on PatchGAN, so that the repaired pod image is infinitely close to the complete and unobstructed pod image, thereby solving the problem of mutual occlusion between pods or missing pod images. Finally, the trait extraction method is used to complete the acquisition of the trait information of each pod. The system and method provided by the present invention can greatly improve the statistical accuracy and efficiency of soybean pod phenotypic traits, avoiding the subjective errors caused by traditional manual statistics;

[0027] (2) In the repair system, method and pod trait extraction method for seed pod occlusion created by the present invention, the residual convolution module effectively solves the gradient explosion and gradient vanishing problems by adding the splicing features obtained by the first branch to the residual connection operation of the second branch, thereby promoting the convergence of the model; in addition, the traditional U-Net network uses a traditional downsampling operation on the feature map in the encoding branch, which will lead to the loss of fine-grained information and the learning of ineffective feature representation, resulting in lower resolution of smaller targets themselves, and limited context information for model learning. The present invention uses SPD-Conv operation to perform lossless downsampling operation on pixels to retain as much small target information as possible;

[0028] (3) In the repair system, method and pod trait extraction method for detecting pod occlusion created by the present invention, a feedforward network is introduced into the self-attention module to process the attention features. Through simple spatial attention and gated linear units, the channel information and spatial information of the input features are enhanced, while reducing the number of model parameters and enhancing its expression ability, allowing the repair system to learn more complex feature representations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0030] Figure 1 This is an overall schematic diagram of a repair system for seed pod obstruction according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of a pod repair generator according to an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a residual convolution module according to an embodiment of the present invention;

[0033] Figure 4 A schematic diagram of a self-attention module according to an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of a pod repair discriminator according to an embodiment of the present invention;

[0035] Figure 6 A schematic flow chart of a method for repairing seed pod obstruction according to an embodiment of the present invention;

[0036] Figure 7 This is a repair effect diagram of the embodiment of the present invention;

[0037] Figure 8 A schematic diagram of the process of extracting pod traits according to an embodiment of the present invention;

[0038] Figure 9 A process diagram of the pod trait extraction method according to an embodiment of the present invention;

[0039] Figure 10 A scatter plot of pod lengths extracted from the restored pod image as described in an embodiment of the present invention;

[0040] Figure 11 A scatter plot of pod widths extracted from the restored pod image as described in an embodiment of the present invention;

[0041] Figure 12 A scatter plot of the pod perimeter extracted from the restored pod image as described in an embodiment of the present invention;

[0042] Figure 13 This is a scatter plot of the pod projection area extracted from the restored pod image as described in the embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0044] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0045] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0046] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0047] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0048] like Figure 1 As shown, the PIX2PIX network model of the repair system for detecting pod occlusion described in the embodiment of the present invention specifically includes a pod repair generator and a pod repair discriminator. The pod repair generator uses an improved U-Net structure to process the defective input pod image to obtain the corresponding pod repair image. The pod repair generator is provided with a multi-level residual convolution module that performs multi-level feature extraction and enhancement on the input pod image. The pod repair discriminator is designed based on PatchGAN and calculates the difference between the complete image corresponding to the input pod image and the corresponding pod repair image.

[0049] Pods repair generator such as Figure 2 As shown, it includes a peapod feature encoding branch and a peapod feature decoding branch. Since there is a large amount of semantic information in the hidden layer of the model, and the number of feature channels after multiple convolutions is large, and the size of the feature map after multiple downsampling is small, it is suitable to use the self-attention mechanism in the hidden layer to replace the traditional convolution layer. The parallel calculation of self-attention does not bring too much additional computing overhead, and the internal correlation of data and features is captured through the self-attention mechanism. Based on the above considerations, the present invention sets a self-attention module (i.e. Figure 2 In the peapod feature encoding branch, the input peapod image is processed by a multi-level residual convolution module to obtain the peapod encoding feature, and the feature output of each level of residual convolution module is jump-connected to the peapod feature decoding branch. In this embodiment of the present invention, the peapod feature encoding branch includes 7 levels of residual convolution modules (i.e. Figure 2E1~E7 in the self-attention module). In the self-attention module, the peapod encoding feature is continuously subjected to self-attention operations and spatial and channel information enhancement to obtain a peapod enhancement feature. In the peapod feature decoding branch, the peapod enhancement feature is upsampled and decoded step by step in combination with the features output by the peapod feature encoding branch to obtain a peapod repaired image. The peapod feature decoding branch includes a cascade of multiple transposed convolution layers, and the number of transposed convolution layers is consistent with the number of residual convolution modules; the first-level transposed convolution layer upsamples the peapod encoding feature in combination with the peapod enhancement feature; each subsequent transposed convolution layer upsamples the features upsampled by the previous transposed convolution layer in combination with the features of the corresponding scale output by the peapod feature encoding branch, until the last transposed convolution layer outputs the peapod repaired image. In an embodiment of the present invention, the peapod feature decoding branch includes a cascade of 7-level transposed convolution layers (such as Figure 2 D1~D7 in ). The first level transposed convolution layer (i.e. Figure 1 D1 in the figure combines the pod enhancement feature to upsample the pod encoding feature; each subsequent level of transposed convolution layer combines the pod feature encoding branch through a jump connection (such as Figure 2 The dotted line in the figure) outputs the features of the corresponding scale, and upsamples the features after upsampling in the previous level transposed convolution layer until the last level transposed convolution layer (i.e. Figure 1 D7 in ) Output the pod repair image.

[0050] In some embodiments, each level of residual convolution module includes a first branch and a second branch; in the first branch, after performing multiple convolution operations on the input image or input features, the features after each convolution are spliced ​​to obtain spliced ​​features; in the second branch, after performing no less than one convolution operation on the input image or input features, the convolved features are added to the corresponding elements of the spliced ​​features, and the added features are subjected to SPD-Conv operation to complete lossless downsampling to obtain output features.

[0051] In the embodiment of the present invention, the structure of each level of residual convolution module is as follows: Figure 3 As shown in the first branch, the input image or input feature is continuously subjected to three convolution operations with a convolution kernel of 3×3, a stride of 1, and symmetrical padding on the edges, and normalized after each convolution operation (as shown in Figure 3 BN in ) and ReLU activation, concatenating the features after each processing (such as Figure 3, and obtain the concatenated features. In the second branch, the input image or input features are subjected to a continuous convolution operation with a 1×1 kernel and a stride of 1. Normalization and ReLU activation are also performed after the convolution operation. The processed features are then added to the corresponding elements of the concatenated features. The added features are then subjected to an SPD-Conv operation for lossless downsampling to obtain the output features. The SPD-Conv operation involves first performing an SPD (space-to-depth) operation on the features, followed by a convolution operation with a 1×1 kernel and a stride of 1.

[0052] The present invention effectively solves the gradient explosion and gradient vanishing problems by adding the splicing features obtained from the first branch to the residual connection operation of the second branch, thereby promoting the convergence of the model. In addition, the traditional U-Net network's encoding branch uses traditional downsampling operations on feature maps, which leads to the loss of fine-grained information and the learning of ineffective feature representations, resulting in lower resolution of smaller targets themselves and limited contextual information for model learning. The present invention uses SPD-Conv operations to losslessly downsample pixels, preserving as much small target information as possible.

[0053] In the self-attention module of some embodiments, during the process of continuously performing self-attention operations on pod coding features: after performing a channel dimension upgrade operation on the pod coding features, the upgraded features are divided into three parts through a dimension slicing operation to obtain a query matrix, a key matrix and a value matrix; after multiplying the key matrix of the query matrix, a probability matrix is ​​generated through a SoftMax operation, and after multiplying the probability matrix with the value matrix, the dimension is transformed and convolved, and then the pod coding features are added to obtain an attention feature; during the process of strengthening spatial and channel information in the self-attention module: after upgrading the attention feature, the feature is divided into two parts, the two features are respectively subjected to depthwise separable convolution and multiplied, and the multiplied feature is convolved and then added to the attention feature to obtain a pod enhanced feature. The present invention introduces a feedforward network in the self-attention module to process the attention feature. Through simple spatial attention and gated linear units, the model parameters are reduced while its expression ability is enhanced, allowing the repair system to learn more complex feature representations.

[0054] In this embodiment of the present invention, the structure of the self-attention module is as follows: Figure 4As shown, the peapod encoding features are first layer-normalized (LN). Then, depthwise separable convolution is used to triple the channel dimension of the peapod encoding features. The resulting features are then split into three parts, resulting in a query matrix (Q), a key matrix (K), and a value matrix (V). The query matrix is ​​multiplied by the key matrix, followed by a SoftMax operation to generate a probability matrix. After multiplying the probability matrix by the value matrix, the matrix is ​​transformed and convolved with a 1×1 kernel. The matrix is ​​then added to the peapod encoding features to generate the attention features. The attention features are first layer-normalized and then doubled in channel dimension using a 1×1 convolution. The resulting features are then split into two parts, each of which undergoes depthwise separable convolution. A GELU activation is then applied to one of the features after the depthwise separable convolution. The two resulting features are then multiplied, convolved with a 1×1 kernel, and added to the attention features to generate the peapod enhancement features.

[0055] In the pod repair discriminator provided in some embodiments, multiple convolutions are used to replace the image blocking operation in PatchGAN. That is, in the pod repair discriminator, multiple continuous convolution operations are performed on the pod repair image and the corresponding real pod complete image respectively to complete feature extraction. The extracted features are subjected to subsequent image blocking operations in PatchGAN to complete feature comparison, and a authenticity score is given to evaluate the pod repair image. The authenticity score is used to guide the pod repair generator to complete the repair of the input pod image.

[0056] The pod repair discriminator provided by the embodiment of the present invention is as follows Figure 5 As shown, the input image undergoes four consecutive convolution operations to extract features. After each convolution, an activation operation such as LeakyReLU is performed to increase nonlinearity. With the accumulation of convolution operations, the spatial size of the features gradually decreases, the number of channels gradually increases, and the extracted features become increasingly abstract and advanced. The extracted features are then input into the PatchGAN module for comparison and a authenticity score is assigned to evaluate the inpainted image. This authenticity score is then used to guide the inpainting generator to complete the inpainting of the input image.

[0057] The present invention also provides a method for repairing pod obstruction, such as Figure 6 Shown, including:

[0058] A1: Obtain a dataset of pod images and preprocess them to obtain a training set that includes pod occlusion conditions.

[0059] In this embodiment of the present invention, the process of obtaining the training set in step A1 includes:

[0060] The pods of naturally mature soybean plants are removed and randomly scattered on the top of a background board. The camera is used to capture images of unobstructed pods and corresponding obstructed pods. The two images are paired to form a pod image dataset. Since there are many ways to obstruct pods, which are highly random, including but not limited to: no obstruction or slight interference, top obstruction, middle obstruction, tail obstruction, and side obstruction of a single pod; side-by-side obstruction, staggered obstruction of multiple pods, etc., it is difficult to create a dataset. Insufficient simulation of occlusion in the dataset will lead to unknown situations in the system repair, resulting in repair failure, which indirectly leads to the inability to extract phenotypic traits of the pods. Therefore, the present invention considers all possible occlusion situations and images each type of occlusion separately. After obtaining the pod image dataset containing all occlusion situations, the pod image dataset is enhanced to simulate the transformation of pods in various situations and the images of pods in other environments. The data enhancement includes image flipping, image color conversion, and image blurring. Specifically, image flipping simulates the existence of the same occlusion situation at different angles; image color transformation simulates soybean pictures taken in different environments, which includes adjusting the brightness, contrast, saturation, and hue parameters of the soybean pictures to perform the simulation; image blurring simulates the imaging device failing to focus, resulting in blurred imaging.

[0061] By performing data augmentation on the original soybean images, we can simulate various situations of pod occlusion and greatly expand the data set. Each augmentation method provides new data to the repair system, improving its robustness.

[0062] A2: Using the training set obtained in step A1, the repair system for detecting pod occlusion provided by the present invention is trained to obtain a pod repair model.

[0063] During training, the goal of the peapod inpainting discriminator is to maximize the probability of identifying real images as real, while minimizing the probability of identifying generated images as real. Specifically, for two real images, the peapod inpainting discriminator should output a probability close to 1; for two generated images, it should output a probability close to 0. By continuously adjusting system parameters, the peapod inpainting discriminator can accurately distinguish between real and generated images, providing effective feedback to the peapod inpainting generator, encouraging it to produce more realistic images.

[0064] In some embodiments, the pod repair model is trained using the repair objective function in step A2;

[0065] The repair objective function is:

[0066] G'=argminmaxLOSS cGAN (G,D)+λ'LOSS1(G);

[0067] Among them, G' represents the repair objective function, D represents the pod repair discriminator, G represents the pod repair generator, LOSS1 represents the L1 loss function, and λ' represents the repair weight; LOSS cGAN (G,D) represents the repair loss function, which is:

[0068] LOSS cGAN (G,D)=E x,y [logD(x,y)]+E x,z [log(1-D(x,G(x,y)))];

[0069] Among them, x represents the real image, y represents the corresponding constraint, z represents the noise, E x,y [logD(x,y)] represents the expectation of the pod repair discriminator for the real image, E x,z [log(1-D(x,G(x,y)))] represents the expectation of the peapod inpainting discriminator for the image inpainted by the peapod inpainting generator.

[0070] A3: Based on the training results of step A2, adjust the hyperparameters when training the pod repair model until the optimal pod repair model is obtained.

[0071] The hyperparameters in the embodiment of the present invention include: the image size during training is 256×256, the batch size is 16, the initial learning rate is 2e-4, the annealing cosine training strategy is adopted, the final learning rate is reduced to 1e-6, the Adam optimizer is adopted, and the number of iterations is 200.

[0072] A4: Input the pod image to be repaired into the pod repair generator in the optimal pod repair model obtained in step A3 to obtain the corresponding pod repair image.

[0073] The effect of the repair method for seed pod shading provided by the present invention is as follows: Figure 7 As shown, Figure 7 The upper picture of (a) shows the situation where the pods slightly interfere with the unobstructed state. Figure 7 The lower image of (a) shows the corresponding pod repair image; Figure 7 The upper picture in (b) shows the situation where the pod is blocked from the side. Figure 7 The lower image in (b) shows the corresponding pod repair image; Figure 7 The upper picture in (c) shows the situation where multiple pods are blocked side by side. Figure 7 The lower image in (c) shows the corresponding pod repair image.

[0074] from Figure 7It can be seen that the method provided by the present invention can well repair the pod occlusion problem, so that there is only one target pod in the image, and the background is also removed. After obtaining the binary image through traditional CV image processing, the characteristics of the single pod can be well extracted. If the detected image is directly sent to the pod trait extraction, the traditional CV image processing cannot solve the pod occlusion problem. The extracted binary image must be a situation where multiple pods are stuck together, and the characteristics of a single pod cannot be extracted. That is, the method provided by the present invention can well solve the occlusion problem between pods, and accurately remove or repair the overlapping parts of the pods through the repair network, so that only a single pod remains in the repaired image, and the background can also be removed, providing a good foundation for the subsequent single pod trait extraction.

[0075] The present invention also provides a method for extracting pod traits, such as Figure 8 and Figure 9 ,include:

[0076] B1: Obtain the restored bean pod image using the restoration method for bean pod occlusion provided by the present invention. Figure 9 (a) in the.

[0077] B2: Grayscale the bean pod image repaired in step B1, extract the grayscale image of the channel with the largest contrast between the bean pod and the background; use the threshold segmentation algorithm to binarize the grayscale image to obtain the binary image of the bean pod. Figure 9 (b) in the.

[0078] B3: Extract the outer contour of the pod based on the binary image of the pod obtained in step B2, and obtain the pod characteristics based on the outer contour of the pod. Figure 9 (c) in the.

[0079] In some embodiments, the pod traits include pod length, pod width, pod circumference and pod projected area. In the process of obtaining the pod length, two points on the outer contour of the pod are arbitrarily selected, and the distance between the two points is calculated. The maximum value of all distances is the pod length. In the process of obtaining the pod width, the normal slope of the straight line between the two points with the maximum distance is calculated, and then the distance between the intersection of the normal with the same normal slope and the outer contour of the pod at different intercepts is calculated. The maximum value of all distances is the pod width. In the process of obtaining the pod circumference, the sum of the distances between all adjacent pixel points on the outer contour of the pod is the pod circumference. In an embodiment of the present invention, based on the points on the pod contour, if the two points are adjacent in the upper and lower or left and right directions, the distance between the two points is defined as 1. If the two points are adjacent in the upper left, lower left, upper right or lower right directions, the distance between the two points is defined as , the sum of the distances between all adjacent pixels on the contour is the pod perimeter. The pod projected area is the number of pixels within the pod's outer contour.

[0080] The embodiment of the present invention adopts R 2 The (R-squared) coefficient and trait scatter plot were used to evaluate the extraction effect of pod traits. In statistics, R 2 Also known as the coefficient of determination or determination coefficient, it is used to indicate the degree to which the independent variable explains the dependent variable, and its value range is between 0 and 1. 2 The closer the coefficient is to 1, the better the model fits the data; R 2 The closer the coefficient is to 0, the weaker the model's ability to explain the data. 2 The coefficient is 0.9823, and the R of the extracted pod width 2 The coefficient is 0.9783, and the R of the pod perimeter is extracted 2 The coefficient is 0.9569, and the R of the projected area of ​​the pod is extracted 2 The coefficient is 0.9756, and the R 2 The coefficients are all close to 1, indicating that the pod trait extraction method provided by the present invention can effectively and accurately extract the pod traits.

[0081] The scatter plots of various pod characteristics are extracted from the repaired pod image. Figures 10 to 13 As shown, Figure 10 To extract the scatter plot of pod length from the repaired pod image, Figure 11 To extract the scatter plot of pod width from the repaired pod image, Figure 12 To extract the scatter plot of pod perimeter from the repaired pod image, Figure 13 A scatter plot of the pod projection area extracted from the restored pod image. Figure 10 R of the real pod phenotypic data and the phenotypic data restored using the method provided by the present invention 2 The values ​​reflect that the method provided by the present invention can effectively repair the pod occlusion problem, so that the statistical pod phenotypic parameters are very close to the true values. This is because the present invention constructs a better pod repair generator, which can extract the texture and detail information in the pod image more quickly through the residual convolution and self-attention mechanism, so that the generated image is closer to the real result. That is, the method provided by the present invention can well repair the occluded pods, overcoming the tedious process of using traditional CV to extract pod traits, which requires manual separation of individual pods before the traits can be extracted. The present invention can directly repair the pods in the case of pod occlusion, perfectly extract the pod traits, and the results show that the present invention has high accuracy.

[0082] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0083] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A repair system for seed pod obstruction, characterized in that: include: Improve the pod repair generator of the U-Net structure to process the defective input pod image and obtain the corresponding pod repair image; The pod restoration generator is provided with a multi-level residual convolution module for performing multi-level feature extraction and enhancement on the input pod image; A PatchGAN-based pod restoration discriminator calculates the difference between the complete image corresponding to the input pod image and the corresponding pod restoration image; The peapod repair generator includes a peapod feature encoding branch and a peapod feature decoding branch, and a self-attention module is provided between the peapod feature encoding branch and the peapod feature decoding branch; In the peapod feature encoding branch, the input peapod image is continuously processed by a multi-level residual convolution module to obtain a peapod encoding feature, and the features output by the multi-level residual convolution module are jump-connected to the peapod feature decoding branch; each level of the residual convolution module includes a first branch and a second branch; In the first branch, after performing multiple convolution operations on the input image or input features, the features after each convolution are spliced ​​to obtain a spliced ​​feature; In the second branch, after performing a convolution operation on the input image or input feature for at least one consecutive time, the convolved feature is added to the corresponding element of the spliced ​​feature, and the added feature is subjected to an SPD-Conv operation to complete lossless downsampling to obtain an output feature; In the self-attention module, the pod encoding features are continuously subjected to self-attention operations and spatial and channel information enhancement to obtain pod enhancement features; In the peapod feature decoding branch, the peapod enhancement features are upsampled and decoded step by step in combination with the features output by the peapod feature encoding branch to obtain a peapod repaired image.

2. The repair system for seed pod obstruction according to claim 1, characterized in that: During the process of the self-attention module continuously performing self-attention operations on the pod encoding features: After performing a channel dimension upgrade operation on the peapod encoding feature, the upgraded feature is divided into three parts through a dimension slicing operation to obtain a query matrix, a key matrix, and a value matrix. The query matrix and the key matrix are multiplied and a probability matrix is ​​generated through a SoftMax operation. After the probability matrix is ​​multiplied by the value matrix, the matrix is ​​dimensionally transformed and convolved, and then added to the peapod encoding feature to obtain the attention feature. In the process of the self-attention module strengthening spatial and channel information: The attention feature is divided into two parts after dimension upgrading, and the two features are subjected to depthwise separable convolution respectively and then multiplied. The multiplied features are convolved and then added with the attention feature to obtain the pod enhancement feature.

3. The repair system for seed pod obstruction according to claim 1, characterized in that: The pod feature decoding branch includes a cascade of multiple transposed convolutional layers, and the number of the transposed convolutional layers is consistent with the number of the residual convolution modules; The first-level transposed convolutional layer combines the pod enhancement features to upsample the pod encoding features; Each subsequent transposed convolutional layer combines the features of the corresponding scale output by the peapod feature encoding branch to upsample the features upsampled by the previous transposed convolutional layer until the last transposed convolutional layer outputs the peapod repaired image.

4. The repair system for seed pod obstruction according to claim 1, characterized in that: In the pod repair discriminator, multiple continuous convolution operations are performed on the pod repair image and the corresponding real pod complete image to complete feature extraction. The extracted features are subjected to subsequent image block operations in PatchGAN to complete feature comparison, and a authenticity score is given to evaluate the pod repair image. The authenticity score is used to guide the pod repair generator to complete the repair of the input pod image.

5. A method for repairing pod obstruction, characterized in that: include: Obtain a pod image dataset, and preprocess the pod image dataset to obtain a training set including pod occlusion conditions; Using the training set, training the repair system for detecting pod occlusion according to any one of claims 1 to 4 to obtain a pod repair model; According to the training results, the hyperparameters of the pod repair model are adjusted until the optimal pod repair model is obtained; The pod image to be repaired is input into the pod repair generator in the optimal pod repair model to obtain the corresponding pod repair image.

6. The method for repairing seed pod obstruction according to claim 5, characterized in that: In the process of training the repair system, the pod repair model is trained using a repair objective function; The repair objective function is: <h2 style=";text-align:left;direction:ltr">G'=argminmaxLOSS<h2 style=";text-align:left;direction:ltr"> cGAN <h2 style=";text-align:left;direction:ltr"> (G,D)+λ'LOSS1(G); Where G' represents the repair objective function, D represents the pod repair discriminator, G represents the pod repair generator, LOSS1 represents the L1 loss function, and λ' represents the repair weight; LOSS cGAN (G,D) represents the repair loss function, which is: LOSS cGAN (G,D)=E x,y [logD(x,y)]+E x,z [log(1-D(x,G(x,y)))]; Among them, x represents the real image, y represents the corresponding constraint, z represents the noise, E x,y [logD(x,y)] represents the expectation of the pod restoration discriminator for the real image, E x,z [log(1-D(x,G(x,y)))] represents the expectation of the pod restoration discriminator for the image restored by the pod restoration generator.

7. A method for extracting pod traits, characterized in that: include: Obtaining a repaired bean pod image using the repair method for detecting bean pod occlusion according to any one of claims 5 or 6; Grayscale the restored pod image and extract the grayscale image of the channel with the largest contrast between the pod and the background; Binarize the grayscale image using a threshold segmentation algorithm to obtain a pod binary image; The outer contour of the pod is extracted based on the binary image of the pod, and the pod traits are obtained based on the outer contour of the pod.

8. The method for extracting pod traits according to claim 7, wherein The pod traits include pod length, pod width, pod circumference and pod projected area.

9. The method for extracting pod traits according to claim 8, characterized in that In the process of obtaining the length of the pod, two points on the outer contour of the pod are randomly selected and the distance between the two points is calculated. The maximum value of all distances is the length of the pod. In the process of obtaining the pod width, the slope of the normal line between the two points with the maximum distance is calculated, and then the distances of the intersections of the normal lines with the same normal slope and different intercepts with the outer contour of the pod are calculated. The maximum value of all the distances is the pod width. In the process of obtaining the perimeter of the pod, the sum of the distances between all adjacent pixel points on the outer contour of the pod is the perimeter of the pod; The pod projection area is the number of all pixels within the outer contour of the pod.

Images