Methods, apparatus, equipment and media for detecting soybean plant leaf area index

Abstract

This application relates to a method, apparatus, device, and medium for detecting the leaf area index (LAI) of soybean plants. The method includes: determining the vertical projected area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region; calculating the product between the volume density map and the mask image from a view perpendicular to the ground to determine the pixel values ​​of all leaves of the soybean plant; determining the sum of the vertical projected areas of all leaves in an unobstructed state based on the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region; and determining the LAI of the soybean plant based on the sum of the vertical projected areas of all leaves in an unobstructed state and the vertical projected area of ​​the soybean plant. This application can significantly reduce costs while greatly improving the detection accuracy of the LAI.

Description

Technical Field

[0001] This application relates to the field of agricultural production, and in particular to a method for detecting the leaf area index of soybean plants, a corresponding device, electronic equipment, and a computer-readable storage medium. Background Technology

[0002] Crop phenotyping technology research is key to accelerating the breeding of high-yield and high-quality varieties and is an important tool for developing modern seed industry. Leaf Area Index (LAI) is an important parameter in crop phenotyping technology. The leaf area index of a single plant is defined as the multiple of the total vertical projection area of ​​all leaves under unshaded conditions on a unit of land area to the total vertical projection area of ​​the plant.

[0003] Traditional methods for calculating leaf area index (LAI) are mostly based on direct measurement using two-dimensional imaging technology. This involves tracing the outline of the leaf to be measured on a specific type of coordinate paper with a pencil, cutting out the coordinate paper according to the leaf shape, weighing the coordinate paper, and calculating the leaf area using a formula. Alternatively, the leaf to be measured can be picked and the leaf area can be calculated using methods such as grid point method or square method. However, when soybean plants have complex morphology and the leaves overlap, these methods are prone to errors, the extraction process is time-consuming, and the information obtained is limited.

[0004] In summary, given the complex morphology of soybean plants and the mutual shading of leaves in existing technologies, methods such as grid point method and square method for calculating leaf area have problems such as being prone to errors, time-consuming extraction process, and limited information extracted. The applicant has made corresponding explorations to solve these problems. Summary of the Invention

[0005] The purpose of this application is to solve the above-mentioned problems by providing a method, device, electronic equipment and computer-readable storage medium for detecting the leaf area index of soybean plants.

[0006] To achieve the various objectives of this application, the following technical solution is adopted:

[0007] A method for detecting the leaf area index of soybean plants, proposed to meet one of the purposes of this application, includes:

[0008] Responding to the soybean plant leaf area index detection command, acquire images of the soybean plants to be detected;

[0009] Based on the neural radiation field model trained to convergence, an implicit representation of the soybean plant to be detected is generated from the image of the soybean plant to be detected. The implicit representation of the soybean plant to be detected is rendered to determine a two-dimensional image and a volume density map from a view perpendicular to the ground.

[0010] The area of ​​each pixel in the soybean leaf region is determined, and the two-dimensional image is segmented based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected. The number of pixels in the soybean leaf region in the mask image is calculated and determined. The vertical projection area of ​​the soybean plant is determined based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region.

[0011] The product between the volume density map and the mask image at the view perpendicular to the ground is calculated to determine the pixel value of all leaves of the soybean plant. Based on the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region, the total vertical projection area of ​​all leaves in the unobstructed state is determined.

[0012] The leaf area index of the soybean plant is determined based on the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0013] Optionally, the step of determining the vertical projection area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region includes:

[0014] The distance between the camera and the soybean plant, the camera focal length, the number of pixels in the soybean leaf area, and the area of ​​each pixel in the soybean leaf area were obtained during the shooting.

[0015] Calculate and determine a first ratio between the distance between the camera and the soybean plant at the time of shooting and the focal length of the camera;

[0016] Calculate the first product between the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0017] The product between the first ratio and the first product is calculated to determine the vertical projected area of ​​the soybean plant.

[0018] Optionally, the step of determining the sum of the vertical projection areas of all leaves in the unobstructed state based on the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region includes:

[0019] The system obtains the distance between the camera and the soybean plant, the camera's focal length, the pixel values ​​of all the soybean plant's leaves, and the area of ​​each pixel in the soybean leaf region during the shooting process.

[0020] Calculate and determine a second ratio between the distance between the camera and the soybean plant at the time of the shot and the focal length of the camera;

[0021] Calculate the second product between the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region;

[0022] Calculate the product between the second ratio and the second product to determine the total vertical projected area of ​​all leaves in the unobstructed state.

[0023] Optionally, the step of determining the leaf area index of the soybean plant based on the sum of the vertical projected areas of all leaves in the unshaded state and the vertical projected area of ​​the soybean plant includes:

[0024] Determine the vertical projected area of ​​the soybean plant and the total vertical projected area of ​​all leaves under unobstructed conditions;

[0025] The leaf area index of the soybean plant is determined by calculating the third ratio between the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant.

[0026] Optionally, the steps of generating an implicit representation of the soybean plant to be detected based on the image of the soybean plant to be detected using a neural radiation field model trained to convergence, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map at a view perpendicular to the ground, include:

[0027] An image of a soybean plant to be detected is input into a neural radiation field model that has been trained to convergence, generating an implicit representation of the soybean plant to be detected, wherein the implicit representation includes the three-dimensional information, color, and density distribution of the soybean plant.

[0028] Based on the implicit representation of the soybean plant to be detected, a two-dimensional image perpendicular to the ground view is rendered using a projection method.

[0029] By calculating the volume density of each point from a vertical perspective, the density value of each pixel is generated to determine the volume density map from a perspective perpendicular to the ground.

[0030] Optionally, the step of segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the plant to be detected includes:

[0031] The preset image segmentation model is invoked, and the two-dimensional image from the perspective perpendicular to the ground is input into the image segmentation model that has been trained to convergence.

[0032] The two-dimensional image viewed from a perspective perpendicular to the ground is segmented to output a masked image containing soybean leaves.

[0033] Optionally, the basic network architecture of the image segmentation model is the SAM model.

[0034] A soybean plant leaf area index detection device provided for another purpose of this application includes:

[0035] The image acquisition module is configured to acquire images of soybean plants to be detected in response to soybean plant leaf area index detection commands.

[0036] The three-dimensional reconstruction module is configured to generate an implicit representation of the soybean plant to be detected based on the image of the soybean plant to be detected using a neural radiation field model that has been trained to convergence, and to render the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map at a view perpendicular to the ground.

[0037] The vertical projection area determination module is configured to determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region.

[0038] The total projection area determination module is configured to calculate the product between the volume density map and the mask image from the perspective perpendicular to the ground to determine the pixel value of all leaves of the soybean plant, and determine the total vertical projection area of ​​all leaves in the unobstructed state based on the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region.

[0039] The leaf area index detection module is configured to determine the leaf area index of the soybean plant based on the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0040] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the soybean plant leaf area index detection method of this application.

[0041] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the soybean plant leaf area index detection method, which, when called by a computer, executes the steps included in the corresponding method.

[0042] Compared to existing technologies, this application addresses the problems of error-proneness, time-consuming extraction processes, and limited information extraction associated with existing methods for calculating leaf area using grid-based or square-based methods when soybean plants have complex morphologies and leaves overlap. This application offers the following advantages, including but not limited to:

[0043] This application utilizes neural radiation field technology to create a three-dimensional model of soybean plants and calculates the leaf area index of the plants based on the three-dimensional model. This method can significantly reduce costs while greatly improving the detection accuracy of the leaf area index.

[0044] Furthermore, this application can significantly improve the detection accuracy of leaf area index (LAI) even when soybean plants have complex morphology and leaves are overlapping each other. By improving the detection accuracy of LAI, the leaf area distribution of soybean plants can be analyzed accurately and without error, providing useful information in ecological and agricultural research, and avoiding problems such as incorrect LAI calculation, time-consuming extraction process, and limited information obtained. Attached Figure Description

[0045] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0046] Figure 1 This is a flowchart illustrating the soybean plant leaf area index detection method in the embodiments of this application;

[0047] Figure 2 This is a schematic diagram illustrating the process of determining a two-dimensional image and a volume density map from a perspective perpendicular to the ground in this application embodiment;

[0048] Figure 3 This is a schematic diagram illustrating the process of obtaining the mask image of soybean leaves in the plant to be detected in an embodiment of this application;

[0049] Figure 4 This is a flowchart illustrating the process of determining the vertical projection area of ​​a soybean plant in an embodiment of this application.

[0050] Figure 5 This is a flowchart illustrating the process of determining the total vertical projected area of ​​all leaves in an unobstructed state in an embodiment of this application.

[0051] Figure 6 This is a flowchart illustrating the process of determining the leaf area index of soybean plants in an embodiment of this application.

[0052] Figure 7 This is a schematic diagram of the soybean plant leaf area index detection device in the embodiments of this application;

[0053] Figure 8 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0054] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0055] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application’s specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when this application states that an element is “connected” or “coupled” to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein may include wireless connection or wireless coupling. The term “and / or” as used herein includes all or any unit and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0057] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0058] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0059] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0060] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0061] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0062] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0063] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0064] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0065] Based on the above exemplary scenarios, please refer to Figure 1 In one embodiment of the soybean plant leaf area index detection method of this application, the method includes:

[0066] Step S10: Respond to the soybean plant leaf area index detection command and acquire the image of the soybean plant to be detected;

[0067] The computing terminal device can respond to the soybean plant leaf area index detection command and acquire the image of the soybean plant to be detected, wherein the image of the soybean plant to be detected can be acquired based on the binocular camera in the UAV.

[0068] Step S20: Based on the neural radiation field model trained to convergence, generate an implicit representation of the soybean plant to be detected according to the image of the soybean plant to be detected, and render the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map at a view perpendicular to the ground.

[0069] After acquiring the image of the soybean plant to be detected, an implicit representation of the soybean plant to be detected is generated based on the neural radiation field model that has been trained to convergence. The implicit representation of the soybean plant to be detected is then rendered to determine a two-dimensional image and a volume density map from a perspective perpendicular to the ground.

[0070] In some embodiments, a volume density map shows the density distribution of each voxel at a view perpendicular to the ground. Brightness or color in the image typically represents different density values, with high-density areas appearing as brighter or darker areas, and low-density areas as darker or lighter areas.

[0071] For further details, please refer to Figure 2 The steps of generating an implicit representation of the soybean plant to be detected based on the image of the soybean plant to be detected using a neural radiation field model trained to convergence, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map at a view perpendicular to the ground, include:

[0072] Step S201: Input the image of the soybean plant to be detected into the neural radiation field model that has been trained to convergence state to generate an implicit representation of the soybean plant to be detected, wherein the implicit representation includes the three-dimensional information, color and density distribution of the soybean plant.

[0073] Step S202: Based on the implicit representation of the soybean plant to be detected, render a two-dimensional image from a perspective perpendicular to the ground using a projection method;

[0074] Step S203: By calculating the volume density of each point under the vertical view, the density value of each pixel is generated to determine the volume density map under the view perpendicular to the ground.

[0075] Specifically, the high-quality reconstruction of soybean plants based on neural radiation fields consists of three stages: data preparation, scene representation, and volume rendering. In the data preparation stage, multi-view images are preprocessed into the data format required by the neural radiation field model. In the scene representation stage, the data from the data preparation process is represented as a three-dimensional scene. In the volume rendering stage, classic volume rendering techniques are used to project the data obtained in the scene representation stage into two-dimensional images. Loss functions are constructed to compare the generated depth image with the true depth image and the generated color image with the true color image. Furthermore, to address the problem of unclear soybean plant texture reconstruction, an L... trans Regularization is used for constraints; to better improve the reconstruction quality, this application introduces a volume density regularization constraint where the weight of sampling points other than the scene surface is 0, and the weight of sampling points on the scene surface is 1.

[0076] The data preparation stage involves preprocessing multi-view images into the data format required by the model. First, multi-view images of soybean plants are captured non-contactly using a camera. Then, the camera pose is obtained using a structure-of-motion (SFM) algorithm, specifically by estimating the pose using the SFM module in COLMAP. The SFM module in COLMAP employs an incremental reconstruction method, first selecting unordered images for feature matching and then optimizing the matching based on geometric conditions. Triangulation is used to recover the sparse structure of the point cloud and perform relative pose estimation. Bundle adjustment is then used to optimize and adjust the data structure, finally outputting the camera's intrinsic parameters and pose. In this process, 1024 sampling points are selected at equal intervals for each ray. Assuming there are n rays in the scene reconstruction, 1024*n positional information (x, y, z) and 1024*n visual information (d) can be obtained.

[0077] The scene representation stage is a process of implicitly representing data as a 3D scene. In this process, the positional information (x, y, z) obtained during data preprocessing is first encoded into higher-dimensional data and then input into an AML (Analog-Multilayer Perceptron) to output volume density and positional features. Then, the positional features and the encoded high-dimensional visual information (d) are input together into a BML to output color information. Specifically, the positional information is first input into the AML, which consists of eight fully connected layers with ReLU, each with 256 channels. A skip connection is used in the fifth ReLU layer to input the positional information along with the data. After the eighth ReLU layer, a fully connected layer without a ReLU function is connected, outputting the volume density and positional feature values. Secondly, the positional features and encoded high-dimensional visual information (d) are input into a fully connected layer without a ReLU function, followed by a fully connected layer with a ReLU function, and finally outputting color information. Color information and volume density values ​​simulate the distribution and transmission of light. Volume density, also known as opacity, simulates light transmission; its value is independent of visual information. A higher volume density indicates a greater probability of light being blocked when a ray reaches that location.

[0078] The volume rendering stage is a process of projecting a 3D scene onto a 2D image. In this process, classic volume rendering techniques are first used to render a batch of color maps and corresponding depth maps based on the volume density and color information obtained in the scene representation stage. The color map rendering formula is as follows:

[0079]

[0080] in, σ i δ represents the volume density value of each sampling point obtained in the scene representation. i c is the distance between two adjacent sampling points. i N represents the color value of each sampling point obtained in the scene representation; N is the number of sampling points.

[0081] The depth map rendering formula is expressed as follows:

[0082]

[0083] The expression for the loss between the depth map obtained in the above process and the true depth map is constructed as follows:

[0084]

[0085] The expression for the loss between the color image obtained in the above process and the true color image is as follows:

[0086]

[0087] The true value of the depth map is obtained through the DPT model.

[0088] Meanwhile, in order to make the textures of the 3D scene clearer, this application introduces L trans Regularization applies constraints, and the calculation formula is as follows:

[0089] L trans =||T(t)·m(t)||2,

[0090] Where T(t) is the opacity, i.e., the probability of light passing through this sampling point; m(t) is the mask indicator, i.e., 0 when t < Z(r) and 1 when t >= Z(r), where t is the position of each sampling point and Z(r) is the coordinate value corresponding to the depth value obtained by the DPT model. Furthermore, to better improve the reconstruction quality of the plants, this application introduces a volume density regularization constraint where the weight of sampling points other than those on the scene surface is 0, and the weight of sampling points on the scene surface is 1. The calculation formula for the regularization constraint is as follows:

[0091]

[0092] Where R is the number of rays used in each batch during each rendering process, N is the number of sampling points on each ray, and w r,i The weight value for each ray is the product of the volume density value and the opacity at each sampling point, i.e., σ. r,i *T r,i .

[0093] Throughout the training process, the aforementioned loss function needs to be backpropagated for model optimization, i.e., the loss function of the entire reconstruction module, which is expressed as follows:

[0094] L rec =L color +L depth +L trans +L w ,

[0095] This embodiment demonstrates that a connection between two-dimensional images and three-dimensional graphics is established through neural radiation field technology. To better address issues such as floating objects and artifacts that occur during existing model reconstruction, volume density-based regularization is introduced for constraint, and L is also introduced. trans Regularization is used for constraints, while high-quality 3D scene reconstruction is based on the calculation of leaf area index using neural radiation fields.

[0096] In some embodiments, the DPT (Dense Prediction Transformer) model is a modern model for depth estimation that leverages the Transformer architecture for high-quality depth map prediction. The DPT model combines the global context modeling capabilities of the Transformer with the local feature extraction of convolutional networks, effectively handling depth estimation tasks at different scales and in complex scenes. Its main features include:

[0097] Global Context Modeling: DPT uses the Transformer mechanism to capture global information in the image, which helps improve the detail and accuracy of the depth map.

[0098] Hybrid Convolution and Transformer: The model combines convolutional neural networks (CNN) and Transformer networks, using convolution to extract local features while using Transformer to handle long-range dependencies.

[0099] High-precision depth prediction: DPT can provide high-precision depth estimation on a variety of benchmark datasets, especially in complex and textured scenes.

[0100] Step S30: Determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on the pre-trained image segmentation model to obtain a mask image of soybean leaves in the plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region.

[0101] Based on a neural radiation field model trained to convergence, an implicit representation of the soybean plant to be detected is generated from the image of the soybean plant to be detected. After rendering the implicit representation of the soybean plant to be detected to determine the two-dimensional image and the volume density map at the view perpendicular to the ground, the area of ​​each pixel in the soybean leaf region is determined. The two-dimensional image is segmented based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected. The number of pixels in the soybean leaf region in the mask image is calculated and determined. The vertical projection area of ​​the soybean plant is determined based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region. The basic network architecture of the image segmentation model is the SAM model.

[0102] In some embodiments, the SAM (Segment Anything Model) is an advanced image segmentation model designed to handle various image segmentation tasks and capable of accurately segmenting any object. It offers advantages such as versatility and flexibility, support for interactive segmentation, and efficient image segmentation, making it particularly suitable for segmenting two-dimensional images of plants to be detected in this application to obtain mask images of soybean leaves within the detected plants.

[0103] For further details, please refer to Figure 3 The step of segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the plant to be detected includes:

[0104] Step S3001: Call the preset image segmentation model and input the two-dimensional image from the perspective perpendicular to the ground into the image segmentation model that has been trained to convergence.

[0105] Step S3002: Perform image segmentation on the two-dimensional image from a perspective perpendicular to the ground, and output a mask image containing soybean leaves.

[0106] For further details, please refer to Figure 4 The step of determining the vertical projection area of ​​a soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region includes:

[0107] Step S301: Obtain the distance between the camera and the soybean plant, the camera focal length, the number of pixels in the soybean leaf area, and the area of ​​each pixel in the soybean leaf area during shooting;

[0108] Step S302: Calculate and determine the first ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera;

[0109] Step S303: Calculate the first product between the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0110] Step S304: Calculate the product between the first ratio and the first product to determine the vertical projected area of ​​the soybean plant.

[0111] Specifically, first, we construct the proportional relationship between the real world and the two-dimensional image. Assuming the plant's height in the two-dimensional image is h, its height in the real world is H, the camera's focal length is f, and the distance between the camera and the soybean plant during shooting is d, we can establish the relationship between the real world and the two-dimensional image based on the pinhole imaging principle. Its expression is as follows:

[0112]

[0113] In the 3D reconstruction process, after the soybean plant is reconstructed in 3D, a 2D image perpendicular to the ground view is rendered, denoted as I. Then, the 2D image I is input into the SAM image segmentation model for segmentation, obtaining a mask image of the soybean leaves. The number of pixels in the soybean leaf region of the mask image is calculated, denoted as num. Assuming the area of ​​each pixel is s, the projected area of ​​the soybean plant on the 2D image is num·s. From the relationship between the real world and the 2D image obtained in the first step, the vertical projected area of ​​the soybean plant in the real world can be obtained, and its calculation formula is as follows:

[0114]

[0115] Where num is the number of pixels in the soybean leaf region of the mask image, s is the area of ​​each pixel in the soybean leaf region, and S projection This represents the vertical projection area of ​​the soybean plant.

[0116] Step S40: Calculate the product between the volume density map and the mask image from the perspective perpendicular to the ground to determine the pixel value of all leaves of the soybean plant. Based on the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region, determine the total vertical projection area of ​​all leaves in the unobstructed state.

[0117] After determining the vertical projection area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region, the product between the volume density map perpendicular to the ground view and the mask image is calculated to determine the pixel value of all leaves of the soybean plant. The total vertical projection area of ​​all leaves in the unobstructed state is determined based on the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region.

[0118] For further details, please refer to Figure 5 The step of determining the sum of the vertical projection areas of all leaves in an unobstructed state based on the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region includes:

[0119] Step S401: Obtain the distance between the camera and the soybean plant, the camera focal length, the pixel values ​​of all leaves of the soybean plant, and the area of ​​each pixel in the soybean leaf region during shooting.

[0120] Step S402: Calculate and determine the second ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera;

[0121] Step S403: Calculate the second product between the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region;

[0122] Step S404: Calculate the product between the second ratio and the second product to determine the total vertical projected area of ​​all leaves in the unobstructed state.

[0123] Specifically, the product between the volume density map perpendicular to the ground view and the mask map obtained from the SAM model segmentation is calculated to obtain the peak value of all leaves of the soybean plant. The peak value of all leaves of the soybean plant represents the pixel value of all leaves of the soybean plant, denoted as num_1. That is, num_1 is the pixel value of all leaves of the soybean plant. Therefore, the sum of the vertical projected areas of all leaves in the unobstructed state can be obtained. The calculation formula is expressed as follows:

[0124]

[0125] Where num_1 represents the pixel values ​​of all leaves of the soybean plant, s represents the area of ​​each pixel in the soybean leaf region, and S all This represents the total vertical projected area of ​​all leaves in the unobstructed state.

[0126] In some embodiments, the total vertical projected area of ​​all leaves in an unobstructed vertical view is calculated. This calculation utilizes the viewpoint-independent nature of the density value during the reconstruction process. The sum of the areas of all leaves is calculated by measuring the peak value of the density. Specifically, the density value reaches its peak when a ray passes through the object's surface; two peaks occur when passing through one leaf, and three peaks occur when passing through two leaves. Since each leaf pixel has a background, the number of density peaks for each ray is the number of leaves passed through plus one. Simultaneously, because some noise is unavoidable during reconstruction, the density map of each ray needs to be smoothed to obtain a shape that is relatively easy for computer terminals to calculate.

[0127] Step S50: Determine the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0128] After determining the sum of the vertical projection areas of all leaves in the unobstructed state based on the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region, the leaf area index of the soybean plant is determined based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0129] For further details, please refer to Figure 6 The step of determining the leaf area index of a soybean plant based on the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant includes:

[0130] Step S501: Determine the vertical projection area of ​​the soybean plant and the total vertical projection area of ​​all leaves under unobstructed conditions.

[0131] Step S502: Calculate and determine the third ratio between the sum of the vertical projected areas of all leaves in the unobstructed state and the vertical projected area of ​​the soybean plant, so as to determine the leaf area index of the soybean plant.

[0132] Specifically, the leaf area index (LAI) of a single soybean plant is defined as the ratio of the total vertical projection area of ​​all leaves in an unshaded state per unit land area to the vertical projection area of ​​the soybean plant. Combining the above process, the soybean LAI can be obtained, and its calculation formula is as follows:

[0133]

[0134] Among them, S all S represents the total vertical projected area of ​​all leaves in the unobstructed state. projection This represents the vertical projection area of ​​the soybean plant.

[0135] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems of error-proneness, time-consuming extraction process, and limited information extraction in calculating leaf area using grid methods and square methods when soybean plants have complex morphology and leaves overlap. This application has, but is not limited to, the following beneficial effects:

[0136] This application utilizes neural radiation field technology to create a three-dimensional model of soybean plants and calculates the leaf area index of the plants based on the three-dimensional model. This method can significantly reduce costs while greatly improving the detection accuracy of the leaf area index.

[0137] Furthermore, this application can significantly improve the detection accuracy of leaf area index (LAI) even when soybean plants have complex morphology and leaves are overlapping each other. By improving the detection accuracy of LAI, the leaf area distribution of soybean plants can be analyzed accurately and without error, providing useful information in ecological and agricultural research, and avoiding problems such as incorrect LAI calculation, time-consuming extraction process, and limited information obtained.

[0138] Please see Figure 7 A soybean plant leaf area index detection device, provided to meet one of the purposes of this application, includes an image acquisition module 1100. The image acquisition module 1100 is configured to acquire an image of a soybean plant to be detected in response to a soybean plant leaf area index detection command; a three-dimensional reconstruction module 1200 is configured to generate an implicit representation of the soybean plant to be detected based on a neural radiation field model trained to convergence, and render the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map from a perspective perpendicular to the ground; a vertical projection area determination module 1300 is configured to determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, and calculate the number of pixels in the soybean leaf region in the mask image. The system comprises: a vertical projection area determination module 1400, which calculates the product between the volume density map and the mask image from a ground-perpendicular viewpoint to determine the pixel values ​​of all leaves of the soybean plant, and determines the total vertical projection area of ​​all leaves in the unobstructed state based on the pixel values ​​of all leaves and the area of ​​each pixel in the soybean leaf region; and a leaf area index detection module 1500, which determines the leaf area index of the soybean plant based on the total vertical projection area of ​​all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, thereby completing the detection of the leaf area index of the soybean plant.

[0139] Based on any embodiment of this application, please refer to Figure 8 Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 8The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, the processor can implement a soybean plant leaf area index detection method. The processor of the computer device provides computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the soybean plant leaf area index detection method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] In this embodiment, the processor is used to execute... Figure 7 The system contains the specific functions of each module and its sub-modules, and the memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the soybean plant leaf area index detection device of this application, and the server can call the server's program code and data to execute the functions of all sub-modules.

[0141] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the soybean plant leaf area index detection method described in any embodiment of this application.

[0142] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the soybean plant leaf area index detection method described in any embodiment of this application.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0144] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

[0145] In summary, this application can significantly improve the detection accuracy of leaf area index even when soybean plants have complex morphology and leaves that overlap. By improving the detection accuracy of leaf area index, the leaf area distribution of soybean plants can be analyzed accurately and reliably, providing useful information in ecological and agricultural research, and avoiding problems such as errors in leaf area index calculation, time-consuming extraction process, and limited information obtained.

Claims

1. A method for detecting the leaf area index of soybean plants, characterized in that, include: Responding to the soybean plant leaf area index detection command, acquire images of the soybean plants to be detected; Based on the neural radiation field model trained to convergence, an implicit representation of the soybean plant to be detected is generated from the image of the soybean plant to be detected. The implicit representation of the soybean plant to be detected is then rendered to determine a two-dimensional image and a volume density map from a perspective perpendicular to the ground. The area of ​​each pixel in the soybean leaf region is determined, and the two-dimensional image is segmented based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the soybean plant to be detected. The number of pixels in the soybean leaf region in the mask image is calculated and determined. The vertical projection area of ​​the soybean plant is determined based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region. The product between the volume density map and the mask image at the view perpendicular to the ground is calculated to determine the pixel value of all leaves of the soybean plant. Based on the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region, the total vertical projection area of ​​all leaves in the unobstructed state is determined. The leaf area index of the soybean plant is determined based on the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

2. The method for detecting the leaf area index of soybean plants according to claim 1, characterized in that, The step of determining the vertical projection area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region includes: The distance between the camera and the soybean plant, the camera focal length, the number of pixels in the soybean leaf area, and the area of ​​each pixel in the soybean leaf area were obtained during the shooting. Calculate and determine a first ratio between the distance between the camera and the soybean plant at the time of shooting and the focal length of the camera; Calculate the first product between the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region; The product between the first ratio and the first product is calculated to determine the vertical projected area of ​​the soybean plant.

3. The method for detecting the leaf area index of soybean plants according to claim 1, characterized in that, The step of determining the sum of the vertical projection areas of all leaves in the unobstructed state based on the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region includes: The system obtains the distance between the camera and the soybean plant, the camera's focal length, the pixel values ​​of all the soybean plant's leaves, and the area of ​​each pixel in the soybean leaf region during the shooting process. Calculate and determine a second ratio between the distance between the camera and the soybean plant at the time of the shot and the focal length of the camera; Calculate the second product between the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region; Calculate the product between the second ratio and the second product to determine the total vertical projected area of ​​all leaves in the unobstructed state.

4. The method for detecting the leaf area index of soybean plants according to claim 1, characterized in that, The steps for determining the leaf area index of a soybean plant based on the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant include: Determine the vertical projected area of ​​the soybean plant and the total vertical projected area of ​​all leaves under unobstructed conditions; The leaf area index of the soybean plant is determined by calculating the third ratio between the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant.

5. The method for detecting the leaf area index of soybean plants according to claim 1, characterized in that, The steps of generating an implicit representation of the soybean plant to be detected based on the image of the soybean plant to be detected using a neural radiation field model trained to convergence, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map at a view perpendicular to the ground, include: An image of a soybean plant to be detected is input into a neural radiation field model that has been trained to convergence, generating an implicit representation of the soybean plant to be detected, wherein the implicit representation includes the three-dimensional information, color, and density distribution of the soybean plant. Based on the implicit representation of the soybean plant to be detected, a two-dimensional image perpendicular to the ground view is rendered using a projection method. By calculating the volume density of each point from a vertical perspective, the density value of each pixel is generated to determine the volume density map from a perspective perpendicular to the ground.

6. The method for detecting the leaf area index of soybean plants according to claim 1, characterized in that, The steps of segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the soybean plant to be detected include: The preset image segmentation model is invoked, and the two-dimensional image from the perspective perpendicular to the ground is input into the image segmentation model that has been trained to convergence. The two-dimensional image viewed from a perspective perpendicular to the ground is segmented to output a masked image containing soybean leaves.

7. The method for detecting the leaf area index of soybean plants according to any one of claims 1 to 6, characterized in that, The basic network architecture of the image segmentation model is the SAM model.

8. A soybean plant leaf area index detection device, characterized in that, include: The image acquisition module is configured to acquire images of soybean plants to be detected in response to soybean plant leaf area index detection commands. The three-dimensional reconstruction module is configured to generate an implicit representation of the soybean plant to be detected based on the image of the soybean plant to be detected using a neural radiation field model that has been trained to convergence, and to render the implicit representation of the soybean plant to be detected to determine a two-dimensional image and a volume density map at a view perpendicular to the ground. The vertical projection area determination module is configured to determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the soybean plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant based on the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region. The total projection area determination module is configured to calculate the product between the volume density map and the mask image from the perspective perpendicular to the ground to determine the pixel value of all leaves of the soybean plant, and determine the total vertical projection area of ​​all leaves in the unobstructed state based on the pixel value of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region. The leaf area index detection module is configured to determine the leaf area index of the soybean plant based on the sum of the vertical projected areas of all leaves under the unobstructed state and the vertical projected area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images