Soybean plant pod-related phenotype extraction method, device, medium and equipment

Abstract

The invention discloses a soybean plant pod-related phenotype extraction method, device, medium and equipment, and relates to the technical field of agronomy, and the method comprises the following steps: obtaining front and back images of a soybean plant in a to-be-detected area; constructing an improved twin network model; based on an original twin network model, replacing a feature extraction part with a compressed excitation residual network SE-ResNet50, increasing an attention mechanism, and adding a Dropout layer in a network classification layer; inputting the front and back images of the soybean plant into a target detection model to obtain coordinate information of stem nodes; according to the coordinate information of the stem nodes, the coordinate information of the pods is extracted, the Euclidean distance is adopted to obtain the distance between each pod and each stem node, and the pods and the stem nodes are matched; and after the pods are matched with the stem nodes, inputting the soybean plant image with the front and back images containing the pods into the improved twin network model, performing similarity judgment on the pods in the image, and obtaining the soybean plant pod related phenotypes.

Description

Technical Field

[0001] The invention relates to the technical field of agronomy, and in particular to a method, a device, a medium and equipment for extracting soybean plant pod-related phenotypes. Background Art

[0002] Soybean is one of the most important legume seed crops in the world. It is the main source of edible oil, accounting for nearly 25% of the world's total edible oil. The nutritional value of soybean helps prevent heart disease and diabetes to a certain extent. In addition, soybean can be used not only for human consumption, but also for animal consumption, which is the main oil consumption variety in the world. China is the world's fourth largest soybean producer, second only to the United States, Brazil and Argentina. Due to the important role soybean plays in the modern economic environment, large-scale yield evaluation in soybean breeding has become the focus of breeding experts.

[0003] Exploring soybean phenotypes is a key step in breeding high-quality soybeans. Usually, since soybean pods are very dense and there is serious overlap between pods, it is difficult to identify and locate all pods in the image. The pods need to be moved away from the branches to avoid overlap. This method is inefficient, making it difficult to accurately detect soybean pods, which in turn affects the accuracy of extracting soybean plant pod-related phenotypes. Summary of the invention

[0004] The present invention provides a method, device, medium and equipment for extracting soybean plant pod-related phenotypes, which are used to solve the above-mentioned problem existing in the prior art, that is, how to improve the extraction accuracy of soybean pod-related phenotypes in the prior art. The present invention provides a method for extracting soybean plant pod-related phenotypes, which comprises:

[0005] Obtaining images of the front and back sides of soybean plants in the test area;

[0006] Build an improved twin network model; based on the original twin network model, replace the feature extraction part with the compressed excitation residual network SE-ResNet50, add an attention mechanism, and add a Dropout layer to the network classification layer;

[0007] Input the front and back images of soybean plants into the target detection model, identify the stem nodes of the soybean plants, determine the identification results, and obtain the coordinate information of the stem nodes according to the identification results;

[0008] According to the coordinate information of the stem node, the coordinate information of the pod is extracted, and the distance between each pod and each stem node is obtained using the Euclidean distance. The pod and stem node with the minimum distance are selected and matched;

[0009] After all the pods are matched with the corresponding stem nodes, the soybean plant images with pods in both the front and back images are input into the improved Siamese network model to judge the similarity of the pods in the front and back images and obtain the pod-related phenotypes of the soybean plant.

[0010] Optionally, the inputting of the front and back images of the soybean plant into the target detection model, identifying the stem nodes of the soybean plant, and determining the identification result specifically includes:

[0011] The front and back images of the soybean plant are input into the object detection model based on the deep learning network YOLOX to detect the location of the soybean plant stem nodes and give a bounding box as the recognition result.

[0012] Optionally, the method of using the Euclidean distance to obtain the distance between each pod and each stem node specifically includes:

[0013] Use Euclidean distance to get the distance between each pod and each stem node:

[0014]

[0015] Among them, (x1, y1) represents the coordinate value of the pod, (x2, y2) represents the coordinate value of the stem node, and d represents the Euclidean distance between the two coordinate values.

[0016] Optionally, the soybean plant image with both the front and back images containing pods is input into the improved Siamese network model, and similarity judgment is performed on the pods in the front image and the back image, specifically including:

[0017] The soybean plant images with pods in both the front and back images are input into the improved Siamese network model. When the pods in the front and back images meet the set threshold, the pods in the front and back images are the same pods and are saved on the current stem node; otherwise, all pods are matched with the current stem node to determine the pods that match the current stem node.

[0018] Optionally, the obtaining of soybean plant pod-related phenotypes specifically includes:

[0019] Obtain the compactness of the distance between pods of a single soybean plant and the compactness of the nodes;

[0020] The compactness of the distance is obtained using the following formula:

[0021]

[0022] The compactness of the node is obtained using the following formula:

[0023]

[0024] Among them, Compactness dis , Compactness node They represent the compactness of distance and node, respectively. pods is the number of pods on a single soybean plant. pod The node of the soybean plant pod.

[0025] Also includes:

[0026] Before the front and back images of soybean plants are input into the target detection model, data labeling and data enhancement processing are performed on the front and back images of soybean plants.

[0027] The present invention provides a soybean plant pod-related phenotype extraction device, comprising:

[0028] An acquisition module is used to acquire the front and back images of soybean plants in the test area;

[0029] A construction module is used to build an improved twin network model. Based on the original twin network model, the feature extraction part is replaced with the compressed excitation residual network SE-ResNet50, and the attention mechanism is added, and the Dropout layer is added to the network classification layer.

[0030] The recognition module is used to input the front and back images of the soybean plant into the target detection model, identify the stem nodes of the soybean plant, determine the recognition results, and obtain the coordinate information of the stem nodes according to the recognition results;

[0031] A matching module is used to extract the coordinate information of the pods according to the coordinate information of the stem nodes, obtain the distance between each pod and each stem node using the Euclidean distance, select the pod and stem node with the minimum distance, and match the two;

[0032] The extraction module is used to input the soybean plant image containing pods in both the front and back images into the improved twin network model after all the pods are matched with the corresponding stem nodes, judge the similarity of the pods in the front image and the back image, and obtain the pod-related phenotype of the soybean plant.

[0033] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting soybean plant pod-related phenotypes is implemented.

[0034] The present invention provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for extracting soybean plant pod-related phenotypes when executing the program.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a method for extracting soybean plant pod-related phenotypes, which is based on the original twin network model and by adding a SE-ResNet50 network, can better learn the key features in the image, thereby improving the accuracy of obtaining soybean plant pod-related phenotypes, and by adding an attention mechanism for feature extraction, the network can more easily obtain more important feature information; by adding a Dropout layer to the network classification layer of the original twin network model, the overfitting problem can be avoided to construct an improved twin network model; after the pods are matched with the corresponding stem nodes, it is not necessary to remove the pods from the branches, only the front and back images of the soybean plants need to be obtained, and then the soybean plant images containing the pods are input into the improved twin network model to achieve similarity comparison of the pods, accurately detect and count the soybean pods, and then accurately extract the relevant phenotypic characteristics of the soybean pods. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0037] Figure 1 A flowchart of a method for extracting soybean plant pod-related phenotypes provided in an embodiment of the present invention;

[0038] Figure 2 A network structure diagram of an improved twin network model provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of the detection results of image stem nodes using the deep learning network YOLOX provided in an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of the detection results of image pods using the deep learning network YOLOX provided in an embodiment of the present invention;

[0041] Figure 5 A schematic diagram of a computer device for a method for extracting soybean plant pod-related phenotypes provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0044] Figure 1 FIG. 1 is a flow chart of a method for extracting soybean plant pod-related phenotypes provided by an embodiment of the present invention. Figure 1 As shown, this embodiment shows a method for extracting soybean plant pod-related phenotypes, comprising:

[0045] S1: Obtain the front and back images of soybean plants in the test area.

[0046] For example, in this embodiment, soybean plants in the mature stage may be selected as target objects, and front and back images of the soybean plants in the mature stage of the test area may be obtained.

[0047] S2: Build an improved twin network model; based on the original twin network model, replace the feature extraction part with the compressed excitation residual network SE-ResNet50, add the attention mechanism, and add the Dropout layer in the network classification layer.

[0048] like Figure 2 As shown in the figure, in the feature extraction part of the original twin network model, the SE-ResNet50 network is added to extract the features of the soybean plant image, the SE attention mechanism is added to correct the features, and the Dropout layer is added in the network classification layer.

[0049] Exemplarily, the random discarding ratio of Dropout can be selected as 0.2.

[0050] In this step, SE-ResNet50 is used to extract features of soybean plant images. By using the SE attention mechanism to correct the features, the network can more easily obtain more important feature information. By adding a Dropout layer in the network classification layer, the overfitting problem can be avoided.

[0051] S3: Input the front and back images of the soybean plant into the target detection model, identify the stem nodes of the soybean plant, determine the identification results, and obtain the coordinate information of the stem nodes according to the identification results.

[0052] like Figure 3 As shown, by inputting the front and back images of the soybean plant into the target detection model based on the deep learning network YOLOX, the stem nodes of the soybean plant can be detected and the bounding box can be given as the recognition result.

[0053] Similarly, the YOLOX target detection model can be used to detect the location of soybean pods in soybean plant images, such as Figure 4 shown.

[0054] Optionally, the target detection algorithm can be used to identify the stem nodes of mature soybean plants, and the center point of each rectangular box of the detected stem node is taken and saved in a 4-row n-column txt coordinate file, where each column constitutes the area contained in a prediction box. In the actual calculation process, the center point of the prediction box is taken as the position coordinate representing the prediction box and saved in an array of n rows and 2 columns. The specific calculation formula is as follows:

[0055] Node=[Y,X]

[0056] Among them, X represents the horizontal coordinate of the center point, Y represents the vertical coordinate of the center point, and Node represents the n×2 array containing the center points of all predicted boxes.

[0057] In this step, in order to improve the running efficiency of the program and reduce error judgment, after the front and back images of the soybean plant are input into the target detection model to obtain the bounding box, the bounding box can be cropped to save its effective part and eliminate the invalid background.

[0058] S4: Extract the coordinate information of the pods according to the coordinate information of the stem nodes, use the Euclidean distance to obtain the distance between each pod and each stem node, select the pods and stem nodes with the minimum distance, and match the two.

[0059] Optionally, get the distance between each pod and each stem node using Euclidean distance:

[0060]

[0061] Among them, (x1, y1) represents the coordinate value of the pod, (x2, y2) represents the coordinate value of the stem node, and d represents the Euclidean distance between the two coordinate values.

[0062] S5: After all pods are matched with corresponding stem nodes, the soybean plant images with pods in both the front and back images are input into the improved Siamese network model, and the similarity of the pods in the front and back images is judged to obtain the pod-related phenotypes of the soybean plant.

[0063] Optionally, a soybean plant image containing pods in both the front and back images is input into the improved twin network model. When the pods in the front and back images meet a set threshold, the pods in the front and back images are the same pod and are saved on the current stem node; otherwise, all pods are matched with the current stem node to determine the pods that match the current stem node.

[0064] By performing the above steps, the corresponding pods on the stem nodes can be determined to obtain the phenotype of the entire plant.

[0065] For example, the obtained soybean plant pod phenotype may be, for example, the spatial distribution compactness of the whole soybean plant, including the compactness of the distance between pods of a single soybean plant and the compactness of the node position;

[0066] The compactness of the distance is obtained using the following formula:

[0067]

[0068] The compactness of the node is obtained using the following formula:

[0069]

[0070] Among them, Compactness dis , Compactness node They represent the compactness of distance and node, respectively. pods is the number of pods on a single soybean plant. pod The node of the soybean plant pod.

[0071] The above is a method for extracting soybean plant pod-related phenotypes provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for extracting soybean plant pod-related phenotypes, including:

[0072] An acquisition module is used to acquire the front and back images of soybean plants in the test area;

[0073] A construction module is used to build an improved twin network model. Based on the original twin network model, the feature extraction part is replaced with the compressed excitation residual network SE-ResNet50, and the attention mechanism is added, and the Dropout layer is added to the network classification layer.

[0074] The recognition module is used to input the front and back images of the soybean plant into the target detection model, identify the stem nodes of the soybean plant, determine the recognition results, and obtain the coordinate information of the stem nodes according to the recognition results;

[0075] A matching module is used to extract the coordinate information of the pods according to the coordinate information of the stem nodes, obtain the distance between each pod and each stem node using the Euclidean distance, select the pod and stem node with the minimum distance, and match the two;

[0076] The extraction module is used to input the soybean plant image containing pods in both the front and back images into the improved twin network model after all the pods are matched with the corresponding stem nodes, judge the similarity of the pods in the front image and the back image, and obtain the pod-related phenotype of the soybean plant.

[0077] The specific definition of the soybean plant pod-related phenotype extraction device can refer to the definition of the soybean plant pod-related phenotype extraction method above, which will not be repeated here. Each module in the above-mentioned soybean plant pod-related phenotype extraction device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0078] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the soybean plant pod-related phenotype extraction method provided above.

[0079] The present invention also provides Figure 5 The structural diagram of the computer device shown in FIG. Figure 5 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the soybean plant pod-related phenotype extraction method provided in the above embodiment.

[0080] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0081] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for extracting soybean plant pod-related phenotypes, characterized in that: include: Obtain the front and back images of soybean plants in the test area; Construct an improved twin network model; Based on the original twin network model, the feature extraction part is replaced with the compressed excitation residual network SE-ResNet50, and the attention mechanism is added, and the Dropout layer is added to the network classification layer; Input the front and back images of soybean plants into the target detection model, identify the stem nodes of the soybean plants, determine the identification results, and obtain the coordinate information of the stem nodes according to the identification results; According to the coordinate information of the stem node, the coordinate information of the pod is extracted, and the distance between each pod and each stem node is obtained using the Euclidean distance. The pod and stem node with the minimum distance are selected and matched; After all the pods are matched with the corresponding stem nodes, the soybean plant images with pods in both the front and back images are input into the improved Siamese network model to judge the similarity of the pods in the front and back images and obtain the pod-related phenotypes of the soybean plant.

2. The method for extracting soybean plant pod-related phenotypes according to claim 1, characterized in that: The front and back images of the soybean plant are input into the target detection model, the stem nodes of the soybean plant are identified, and the identification result is determined, which specifically includes: The front and back images of the soybean plant are input into the object detection model based on the deep learning network YOLOX to detect the location of the soybean plant stem nodes and give a bounding box as the recognition result.

3. The method for extracting soybean plant pod-related phenotypes according to claim 1, characterized in that: The method of using the Euclidean distance to obtain the distance between each pod and each stem node specifically includes: Use Euclidean distance to get the distance between each pod and each stem node: Among them, (x1, y1) represents the coordinate value of the pod, (x2, y2) represents the coordinate value of the stem node, and d represents the Euclidean distance between the two coordinate values.

4. The method for extracting soybean plant pod-related phenotypes according to claim 1, characterized in that: The method of inputting a soybean plant image in which both the front and back images contain bean pods into the improved twin network model and performing similarity judgment on the bean pods in the front image and the back image specifically includes: The soybean plant images with pods in both the front and back images are input into the improved Siamese network model. When the pods in the front and back images meet the set threshold, the pods in the front and back images are the same pods and are saved on the current stem node; otherwise, all pods are matched with the current stem node to determine the pods that match the current stem node.

5. The method for extracting soybean plant pod-related phenotypes according to claim 1, characterized in that: The step of obtaining the soybean plant pod-related phenotypes specifically includes: Obtain the compactness of the distance between pods of a single soybean plant and the compactness of the nodes; The compactness of the distance is obtained using the following formula: The compactness of the node is obtained using the following formula: Among them, Compactness dis , Compactness node They represent the compactness of distance and node, respectively. pods is the number of pods on a single soybean plant. pod The node of the soybean plant pod.

6. The method for extracting soybean plant pod-related phenotypes according to claim 1, characterized in that: The method further comprises: Before the front and back images of soybean plants are input into the target detection model, data labeling and data enhancement processing are performed on the front and back images of soybean plants.

7. A device for extracting soybean plant pod-related phenotypes, characterized in that: include: An acquisition module is used to acquire the front and back images of soybean plants in the test area; Building module, used to build improved twin network model; Based on the original twin network model, the feature extraction part is replaced with the compressed excitation residual network SE-ResNet50, and the attention mechanism is added, and the Dropout layer is added to the network classification layer; The recognition module is used to input the front and back images of the soybean plant into the target detection model, identify the stem nodes of the soybean plant, determine the recognition results, and obtain the coordinate information of the stem nodes according to the recognition results; A matching module is used to extract the coordinate information of the pods according to the coordinate information of the stem nodes, obtain the distance between each pod and each stem node using the Euclidean distance, select the pod and stem node with the minimum distance, and match the two; The extraction module is used to input the soybean plant image containing pods in both the front and back images into the improved twin network model after all the pods are matched with the corresponding stem nodes, judge the similarity of the pods in the front image and the back image, and obtain the pod-related phenotype of the soybean plant.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting soybean plant pod-related phenotypes according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for extracting soybean plant pod-related phenotypes as claimed in any one of claims 1 to 6 is implemented.

Images