Soybean plant leaf area index detection method, device, equipment and medium

Abstract

The invention relates to a soybean plant leaf area index detection method, device and equipment and a medium. The method comprises the following steps: determining the vertical projection area of a soybean plant according to the number of pixel points in a soybean leaf area and the area of each pixel point in the soybean leaf area; calculating and determining a product between the volume density map and the mask image in a view angle perpendicular to the ground so as to determine pixel values of all leaves of the soybean plant, and determining a sum of vertical projection areas of all the leaves in a non-shielding state according to the pixel values of all the leaves of the soybean plant and the area of each pixel point in the soybean leaf area; and determining the leaf area index of the soybean plant based on the sum of the vertical projection areas of all the leaves in the non-shielding state and the vertical projection area of the soybean plant. According to the invention, the detection precision of the leaf area index can be greatly improved while the cost can be obviously reduced.

Description

Technical Field

[0001] The present application relates to the field of agricultural production, and in particular to a soybean plant leaf area index detection method, a corresponding device, an electronic device and a computer-readable storage medium. Background Art

[0002] Crop phenotyping technology research is the key to accelerate the breeding of high-yield and high-quality varieties, and is an important tool for the development of modern seed industry. Leaf Area Index (LAI) is an important parameter in crop phenotyping technology. The leaf area index of a single plant is defined as the multiple of the sum of the vertical projection area of ​​all unobstructed leaves of the plant per unit land area to the vertical projection area of ​​the plant.

[0003] Traditional leaf area index calculation methods are mostly based on direct measurement using two-dimensional imaging technology, that is, on a specific coordinate paper, the outline of the leaf to be measured is traced with a pencil and the coordinate paper is cut according to the leaf shape, the leaf-shaped coordinate paper is weighed, and the leaf area is calculated according to the formula; or the leaves to be measured are picked and the leaf area is calculated using the grid method, grid method, etc. However, when the soybean plant crop morphology is complex and the leaves cover each other, this type of method is prone to errors, the extraction process is time-consuming, and the extracted information is limited.

[0004] To sum up, when the soybean plant crop morphology is complex and the leaves block each other, the grid method, grid method, etc. used to calculate the leaf area are prone to errors, the extraction process is time-consuming, and the extracted information is limited. The applicant has made corresponding explorations to solve this problem. Summary of the invention

[0005] The purpose of the present application is to solve the above-mentioned problem and to provide a soybean plant leaf area index detection method, corresponding device, electronic device and computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A soybean plant leaf area index detection method proposed to meet one of the purposes of this application includes:

[0008] In response to a soybean plant leaf area index detection instruction, an image of a soybean plant to be detected is obtained;

[0009] generating an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergence state, rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground;

[0010] Determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0011] Calculate and determine the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determine the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf area;

[0012] The leaf area index of the soybean plant is determined based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant to complete the detection of the leaf area index of the soybean plant.

[0013] Optionally, the step of determining the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf area and the area of ​​each pixel in the soybean leaf area includes:

[0014] Obtain the distance between the camera and the soybean plant, the focal length of the camera, the number of pixels in the soybean leaf area, and the area of ​​each pixel in the soybean leaf area during shooting;

[0015] Calculating and determining a first ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera;

[0016] Calculating and determining a first product between the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0017] The product between the first ratio and the first product is calculated to determine the vertical projection area of ​​the soybean plant.

[0018] Optionally, the step of determining the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf region includes:

[0019] Obtain the distance between the camera and the soybean plant, the focal length of the camera, the pixel values ​​of all leaves of the soybean plant, and the area of ​​each pixel point in the soybean leaf area;

[0020] Calculating and determining a second ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera;

[0021] Calculating and determining a second product between the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf region;

[0022] The product between the second ratio and the second product is calculated to determine the sum of the vertical projection areas of all leaves in an unobstructed state.

[0023] Optionally, the step of determining the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant comprises:

[0024] Determine the vertical projection area of ​​the soybean plant and the sum of the vertical projection areas of all leaves in an unobstructed state;

[0025] A third ratio between the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant is calculated to determine the leaf area index of the soybean plant.

[0026] Optionally, the steps of generating an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergent state, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground include:

[0027] Inputting the image of the soybean plant to be detected into the neural radiation field model that has been trained to a convergence state, generating an implicit representation of the soybean plant to be detected, wherein the implicit representation includes three-dimensional information, color, and density distribution of the soybean plant;

[0028] Based on the implicit representation of the soybean plant to be detected, a two-dimensional image perpendicular to the ground is rendered by projection method;

[0029] By calculating the volume density of each point at a vertical viewing angle, a density value for each pixel is generated to determine a volume density map at a viewing angle perpendicular to the ground.

[0030] Optionally, the step of segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the plant to be detected includes:

[0031] Calling a preset image segmentation model, and inputting the two-dimensional image at a viewing angle perpendicular to the ground into the image segmentation model that has been trained to a convergence state;

[0032] The two-dimensional image at a viewing angle perpendicular to the ground is segmented to output a mask image containing soybean leaves.

[0033] Optionally, the basic network architecture of the image segmentation model is a SAM model.

[0034] A soybean plant leaf area index detection device provided for another purpose of the present application comprises:

[0035] An image acquisition module is configured to respond to a soybean plant leaf area index detection instruction and acquire an image of a soybean plant to be detected;

[0036] a three-dimensional reconstruction module, configured to generate an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergence state, and render the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground;

[0037] a vertical projection area determination module, configured to determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0038] a projection area sum determination module, configured to calculate and determine the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determine the vertical projection area sum of all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf area;

[0039] The leaf area index detection module is configured to determine the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0040] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the soybean plant leaf area index detection method described in the present application.

[0041] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the soybean plant leaf area index detection method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0042] Compared with the prior art, the present application aims at the problems that the grid method, the square method and the like are prone to errors, the extraction process is time-consuming, and the extracted information is limited when the soybean plant crop morphology is complex and the leaves block each other in the prior art. The present application includes but is not limited to the following beneficial effects:

[0043] This application is based on the use of neural radiation field technology to perform three-dimensional modeling of soybean plants, and calculates the leaf area index of the plants based on the three-dimensional model, which can significantly reduce costs and greatly improve the detection accuracy of the leaf area index;

[0044] Furthermore, the present application can significantly improve the detection accuracy of the leaf area index when the soybean plant crop morphology is complex and the leaves cover each other. By improving the detection accuracy of the leaf area index, the leaf area distribution of the soybean plant can be accurately analyzed, and useful information can be provided in ecological and agricultural research, avoiding problems such as leaf area index calculation errors, time-consuming extraction process, and limited extracted information. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0046] Figure 1 This is a schematic diagram of the process of detecting the soybean plant leaf area index in the embodiment of the present application;

[0047] Figure 2 A schematic diagram of a process for determining a two-dimensional image and a volume density map at a viewing angle perpendicular to the ground in an embodiment of the present application;

[0048] Figure 3 This is a schematic diagram of the process of obtaining a mask image of soybean leaves in the plant to be detected in an embodiment of the present application;

[0049] Figure 4 This is a schematic diagram of a process for determining the vertical projection area of ​​a soybean plant in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a process for determining the sum of vertical projection areas of all leaves in an unobstructed state in an embodiment of the present application;

[0051] Figure 6 This is a schematic diagram of a process for determining the leaf area index of soybean plants in an embodiment of the present application;

[0052] Figure 7 This is a principle block diagram of a soybean plant leaf area index detection device in an embodiment of the present application;

[0053] Figure 8 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0054] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0055] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when the present application states that an element is "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0057] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0058] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0059] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0060] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0061] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0062] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0063] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0064] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0065] Based on the above exemplary scenarios, please refer to Figure 1 In one embodiment, the soybean plant leaf area index detection method of the present application comprises:

[0066] Step S10, responding to a soybean plant leaf area index detection instruction, acquiring an image of a soybean plant to be detected;

[0067] The computing terminal device can respond to the soybean plant leaf area index detection instruction to obtain the soybean plant image to be detected, wherein the soybean plant image to be detected can be obtained based on the binocular camera in the drone.

[0068] Step S20, generating an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergence state, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground;

[0069] After acquiring an image of a soybean plant to be detected, generating an implicit representation of the soybean plant to be detected based on the image of the soybean plant to be detected based on a neural radiation field model that has been trained to a convergence state, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground;

[0070] In some embodiments, the volume density map shows the density distribution of each voxel at a viewing angle perpendicular to the ground. Brightness or color in an image generally represents different density values, with high-density areas appearing as brighter or darker areas and low-density areas appearing as darker or lighter areas.

[0071] For further information, see Figure 2 The steps of generating an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergent state, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground include:

[0072] Step S201, inputting the soybean plant image to be detected into the neural radiation field model that has been trained to a convergence state, generating an implicit representation of the soybean plant to be detected, wherein the implicit representation includes three-dimensional information, color, and density distribution of the soybean plant;

[0073] Step S202: based on the implicit representation of the soybean plant to be detected, a two-dimensional image perpendicular to the ground is rendered by a projection method;

[0074] Step S203: Generate a density value for each pixel by calculating the volume density of each point at a vertical viewing angle to determine a volume density map at a viewing angle perpendicular to the ground.

[0075] Specifically, the high-quality reconstruction of soybean plants based on neural radiation fields consists of three stages, namely data preprocessing (Data Preparation), scene representation (Scene Representation), and volume rendering (Volume Rendering). In the data preprocessing stage, the present application will preprocess the multi-view images into the data format required by the neural radiation field model. In the scene representation stage, the present application represents the data of the data preprocessing process as a three-dimensional scene. In the volume rendering stage, the present application uses classic volume rendering technology to project the data obtained in the scene representation stage into a two-dimensional image, and constructs a loss function for generating depth images and true depth images, and generating color images and true color images. At the same time, in order to solve the problem of unclear texture reconstruction of soybean plants, L trans Regularization is used for constraint; in order to better improve the reconstruction quality, this application introduces a volume density-based regularization constraint in which the weight of sampling points outside the scene surface is 0, and the weight of sampling points on the scene surface is 1.

[0076] The data preparation stage is the process of preprocessing multi-view images into the data format required by the model. In this process, the multi-view images of soybean plants are first taken by the camera in a non-contact manner, and then the camera's posture is obtained using the motion-based structure recovery algorithm, that is, the SFM module in COLMAP is used to estimate the posture. The SFM module in COLMAP uses an incremental reconstruction method. First, unordered images are selected for feature matching, and matching optimization is performed based on geometric conditions. The sparse structure of the point cloud is restored through triangulation and relative posture estimation is performed. Then, the bundle adjustment method is used to optimize and adjust the data structure, and finally the camera's internal parameters and posture are output. In this process, 1024 sampling points are selected at equal distances for each ray. Assuming that there are n rays in the scene reconstruction, 1024*n position information (x, y, z) and 1024*n visual information (d) can be obtained.

[0077] The scene representation stage is a process of implicitly expressing data into a three-dimensional scene. In this process, the position information (x, y, z) obtained during data preprocessing is first encoded into higher-dimensional data and then input into the multi-layer perceptron (A MLP) to output volume density (density) and position features (feature). The position features (feature) and the encoded high-dimensional visual information (d) are then input into the multi-layer perceptron (B MLP) to output color information (color). Specifically, the position information is first input into the multi-layer perceptron (A MLP), which consists of 8 fully connected layers with Relu, each Relu layer has 256 channels. In the fifth Relu layer, the position information is input into the fifth Relu layer using a skip connection. Then, a fully connected layer without a Relu function is connected after the eighth Relu layer, and the volume density (density) value and the position feature (feature) value are output. Secondly, the position feature (feature) and the encoded high-dimensional visual information (d) are input into a fully connected layer without a Relu function, and then connected to a fully connected layer with a Relu function, and finally the color information (color) is output. The color information (color) and the volume density value (density) simulate the distribution and transmission of light. Volume density, also known as opacity, simulates the transmission of light. Its value is independent of visual information. The larger the volume density, the greater the probability that the ray will be blocked when it traverses to this place.

[0078] The volume rendering stage is a process of projecting a 3D scene into a 2D image. In this process, the volume density and color information obtained in the scene representation stage are firstly rendered using the classic volume rendering technology to obtain a batch of color maps and corresponding depth maps. The color map rendering formula is as follows:

[0079]

[0080] in, σ i is the volume density value (density) of each sampling point obtained in the scene representation; δ i is the distance between two adjacent sampling points; c i is the color value (color) of each sampling point obtained in the scene representation; N is the number of sampling points.

[0081] The depth map rendering formula is expressed as follows:

[0082]

[0083] And construct an expression for the loss between the depth map obtained in the above process and the true depth map, which is expressed as follows:

[0084]

[0085] The expression of the loss between the color map obtained in the above process and the true color map is expressed as follows:

[0086]

[0087] The true value of the depth map is obtained through the DPT model.

[0088] At the same time, in order to make the texture of the three-dimensional scene clearer, this application introduces L trans Regularization is constrained, and the calculation formula is as follows:

[0089] L trans =||T(t)·m(t)|| 2 ,

[0090] Among them, T(t) is the opacity rate, that is, the probability of light passing through this sampling point; m(t) is the mask indicator, that is, it is 0 when t<Z(r), and it is 1 when t>=Z(r), where t is the position of each sampling point, and Z(r) is the coordinate value corresponding to the depth value obtained by the DPT model. In addition, in order to better improve the reconstruction quality of plants, this application introduces a volume density-based regularization constraint that the weight of sampling points other than the scene surface is 0, and the weight of sampling points on the scene surface is 1. The calculation formula of the regularization constraint is expressed as follows:

[0091]

[0092] Among them, R is the number of rays used in batches during each rendering process, N is the number of sampling points on each ray, and w r,i is the weight value on each ray, and the weight value is the product of the volume density value and the opacity of each sampling point, that is, σ r,i *T r,i .

[0093] During the entire training process, the above loss function needs to be back-propagated for model optimization, that is, the loss function of the entire reconstruction module, which is expressed as follows:

[0094] L rec =L color +L depth +L trans +L w ,

[0095] It can be seen from this embodiment that the connection between the two-dimensional image and the three-dimensional graphics is established through the neural radiation field technology. In order to better solve the problems of floating objects and artifacts in the reconstruction process of the existing model, the regularization based on volume density is introduced for constraint and L trans Regularization is performed to constrain, and high-quality 3D scene reconstruction is based on the calculation of the leaf area index based on the neural radiation field.

[0096] In some embodiments, the DPT (Dense Prediction Transformer) model is a modern model for depth estimation that uses the Transformer architecture to perform high-quality depth map prediction. The DPT model combines the global context modeling capabilities of the Transformer with the local feature extraction of the convolutional network, and can effectively handle depth estimation tasks in different scales and complex scenes. Its main features include:

[0097] Global Context Modeling: DPT uses the Transformer mechanism to capture global information in the image, which helps improve the details and accuracy of the depth map.

[0098] Hybrid Convolution and Transformer: The model combines a convolutional neural network (CNN) and a Transformer network, using convolution to extract local features while processing long-distance dependencies through the Transformer.

[0099] High-precision depth prediction: DPT can provide high-precision depth estimation on a variety of benchmark datasets, especially in complex and texture-rich scenes.

[0100] Step S30, determining the area of ​​each pixel in the soybean leaf region, segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, calculating and determining the number of pixels in the soybean leaf region in the mask image, and determining the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0101] Based on the neural radiation field model that has been trained to a convergent state, an implicit representation of the soybean plant to be detected is generated according to the soybean plant image to be detected, and after rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground, the area of ​​each pixel in the soybean leaf region is determined, and the two-dimensional image is segmented based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, the number of pixels in the soybean leaf region in the mask image is calculated and determined, and the vertical projection area of ​​the soybean plant is determined according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region; the basic network architecture of the image segmentation model is a SAM model.

[0102] In some embodiments, the SAM model (Segment Anything Model) is an advanced image segmentation model that is designed to handle various image segmentation tasks and can accurately segment any object. It has the advantages of versatility and flexibility, supports interactive segmentation, and efficient image segmentation, and is particularly suitable for segmenting the two-dimensional image of the plant to be detected in the present application to obtain the mask image of the soybean leaf in the plant to be detected.

[0103] For further information, see Figure 3 The step of segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected comprises:

[0104] Step S3001, calling a preset image segmentation model, and inputting the two-dimensional image at a viewing angle perpendicular to the ground into the image segmentation model that has been trained to a convergence state;

[0105] Step S3002: performing image segmentation on the two-dimensional image at a viewing angle perpendicular to the ground, and outputting a mask image containing soybean leaves.

[0106] For further information, see Figure 4 The step of determining the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf area and the area of ​​each pixel in the soybean leaf area comprises:

[0107] Step S301, obtaining the distance between the camera and the soybean plant, the focal length of the camera, the number of pixels in the soybean leaf area, and the area of ​​each pixel in the soybean leaf area;

[0108] Step S302, calculating and determining a first ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera;

[0109] Step S303, calculating and determining a first product between the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region;

[0110] Step S304: Calculate and determine the product of the first ratio and the first product to determine the vertical projection area of ​​the soybean plant.

[0111] Specifically, first, we construct the proportional relationship between the real world and the two-dimensional image. Assuming that the height of the plant in the two-dimensional image is h, the height in the real world is H, the focal length of the camera is f, and the distance between the camera and the soybean plant during shooting is d, we can establish the relationship between the real world and the two-dimensional image based on the pinhole imaging principle. The expression is as follows:

[0112]

[0113] In the 3D reconstruction process, the soybean plant is reconstructed in 3D and rendered into a 2D image perpendicular to the ground, denoted as I. The 2D image I is then input into the SAM image segmentation model for segmentation to obtain the mask of the soybean leaf, and the number of pixels in the soybean leaf area in the mask is calculated, denoted as num. Assuming that the area of ​​each pixel is s, the projection area of ​​the soybean plant on the 2D image is num·s. Based on the relationship between the real world and the 2D image obtained in the first step, the vertical projection area of ​​the soybean plant in the real world can be obtained, and its calculation formula is as follows:

[0114]

[0115] Among them, num is the number of pixels in the soybean leaf area in the mask image, s is the area of ​​each pixel in the soybean leaf area, and S projection Represents the vertical projection area of ​​soybean plants.

[0116] Step S40, calculating and determining the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determining the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf area;

[0117] After determining the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region, calculating and determining the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determining the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region;

[0118] For further information, see Figure 5 The step of determining the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf region comprises:

[0119] Step S401, obtaining the distance between the camera and the soybean plant, the focal length of the camera, the pixel values ​​of all leaves of the soybean plant, and the area of ​​each pixel point in the soybean leaf area during shooting;

[0120] Step S402, calculating and determining a second ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera;

[0121] Step S403, calculating and determining a second product between the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf region;

[0122] Step S404: Calculate and determine the product of the second ratio and the second product to determine the sum of the vertical projection areas of all leaves in an unobstructed state.

[0123] Specifically, the product of the volume density map at a viewing angle perpendicular to the ground and the mask map obtained by SAM model segmentation is calculated to obtain the peak value of all leaves of the soybean plant. The peak value of all leaves of the soybean plant represents the pixel value of all leaves of the soybean plant, which is recorded as num_1. That is, num_1 is the pixel value of all leaves of the soybean plant. Therefore, the sum of the vertical projection areas of all leaves in the unobstructed state can be obtained. The calculation formula is as follows:

[0124]

[0125] Among them, num_1 is the pixel value of all leaves of soybean plants, s is the area of ​​each pixel in the soybean leaf area, S all Represents the sum of the vertical projection areas of all leaves in the unobstructed state.

[0126] In some embodiments, the sum of the vertical projection areas of all leaves in an unobstructed state at a vertical viewing angle is calculated. In this calculation process, the volume density value (density) in the above-mentioned reconstruction process has the characteristic of being independent of the viewing angle, and the area sum of all leaves is calculated by calculating the peak value of the volume density. That is, when the ray passes through the surface of the object, the volume density value reaches a peak value; if it passes through one leaf, two peak values ​​will appear, and if it passes through two leaves, three peak values ​​will appear. Because each leaf pixel has a background, the number of volume density peak values ​​for each ray is the number of leaves passed by plus one. At the same time, since some noise is still difficult to avoid during the reconstruction process, it is necessary to smooth the volume density map of each ray to obtain a shape that is relatively convenient for the computer terminal device to calculate.

[0127] Step S50, determining the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0128] After determining the total vertical projection areas of all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf area, the leaf area index of the soybean plant is determined based on the total vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant to complete the detection of the leaf area index of the soybean plant.

[0129] For further information, see Figure 6 The step of determining the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant comprises:

[0130] Step S501, determining the vertical projection area of ​​the soybean plant and the sum of the vertical projection areas of all leaves in an unobstructed state;

[0131] Step S502: Calculate and determine a third ratio between the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant to determine the leaf area index of the soybean plant.

[0132] Specifically, the leaf area index of a single soybean plant is defined as the ratio between the sum of the vertical projection areas of all leaves in an unobstructed state per unit land area and the vertical projection area of ​​the soybean plant. That is, combining the above process, the leaf area index of the soybean plant can be obtained, and its calculation formula is as follows:

[0133]

[0134] Among them, S all represents the sum of the vertical projection areas of all leaves in the unobstructed state, S projection Represents the vertical projection area of ​​soybean plants.

[0135] As can be seen from the above embodiments, compared with the prior art, the present application aims at the problems in the prior art such as the grid method, the square method, etc., which are prone to errors, the extraction process is time-consuming, and the extracted information is limited when the soybean plant crop morphology is complex and the leaves block each other. The present application includes but is not limited to the following beneficial effects:

[0136] This application is based on the use of neural radiation field technology to perform three-dimensional modeling of soybean plants, and calculates the leaf area index of the plants based on the three-dimensional model, which can significantly reduce costs and greatly improve the detection accuracy of the leaf area index;

[0137] Furthermore, the present application can significantly improve the detection accuracy of the leaf area index when the soybean plant crop morphology is complex and the leaves cover each other. By improving the detection accuracy of the leaf area index, the leaf area distribution of the soybean plant can be accurately analyzed, and useful information can be provided in ecological and agricultural research, avoiding problems such as leaf area index calculation errors, time-consuming extraction process, and limited extracted information.

[0138] See also Figure 7 , a soybean plant leaf area index detection device provided to meet one of the purposes of the present application, includes an image acquisition module 1100. Among them, the image acquisition module 1100 is configured to respond to the soybean plant leaf area index detection instruction and obtain the soybean plant image to be detected; the three-dimensional reconstruction module 1200 is configured to generate an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergence state, and render the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground; the vertical projection area determination module 1300 is configured to determine the area of ​​each pixel point in the soybean leaf area, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, and calculate and determine the pixel points in the soybean leaf area in the mask image. a number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region, determining the vertical projection area of ​​the soybean plant; a total projection area determination module 1400, configured to calculate and determine the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determining the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel in the soybean leaf region; a leaf area index detection module 1500, configured to determine the leaf area index of the soybean plant based on the total vertical projection area of ​​all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

[0139] Based on any embodiment of this application, please refer to Figure 8 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 8As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a soybean plant leaf area index detection method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the soybean plant leaf area index detection method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In this embodiment, the processor is used to execute Figure 7 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the soybean plant leaf area index detection device of the present application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0141] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the soybean plant leaf area index detection method described in any embodiment of the present application.

[0142] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the soybean plant leaf area index detection method described in any embodiment of the present application.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0144] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0145] In summary, the present application can significantly improve the detection accuracy of the leaf area index when the soybean plant crop morphology is complex and the leaves block each other. By improving the detection accuracy of the leaf area index, the leaf area distribution of the soybean plant can be accurately analyzed, and useful information can be provided in ecological and agricultural research, avoiding problems such as leaf area index calculation errors, time-consuming extraction process, and limited extracted information.

Claims

1. A method for detecting leaf area index of soybean plants, characterized in that: include: In response to a soybean plant leaf area index detection instruction, an image of a soybean plant to be detected is obtained; generating an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergence state, rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground; Determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region; Calculate and determine the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determine the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf area; The leaf area index of the soybean plant is determined based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant to complete the detection of the leaf area index of the soybean plant.

2. The soybean plant leaf area index detection method according to claim 1, characterized in that: The step of determining the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf area and the area of ​​each pixel in the soybean leaf area comprises: Obtain the distance between the camera and the soybean plant, the focal length of the camera, the number of pixels in the soybean leaf area, and the area of ​​each pixel in the soybean leaf area during shooting; Calculating and determining a first ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera; Calculating and determining a first product between the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region; The product between the first ratio and the first product is calculated to determine the vertical projection area of ​​the soybean plant.

3. The soybean plant leaf area index detection method according to claim 1, characterized in that: The step of determining the total vertical projection area of ​​all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf region comprises: Obtain the distance between the camera and the soybean plant, the focal length of the camera, the pixel values ​​of all leaves of the soybean plant, and the area of ​​each pixel point in the soybean leaf area; Calculating and determining a second ratio between the distance between the camera and the soybean plant during the shooting and the focal length of the camera; Calculating and determining a second product between the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf region; The product between the second ratio and the second product is calculated to determine the sum of the vertical projection areas of all leaves in an unobstructed state.

4. The soybean plant leaf area index detection method according to claim 1, characterized in that: The step of determining the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant comprises: Determine the vertical projection area of ​​the soybean plant and the sum of the vertical projection areas of all leaves in an unobstructed state; A third ratio between the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant is calculated to determine the leaf area index of the soybean plant.

5. The soybean plant leaf area index detection method according to claim 1, characterized in that: The steps of generating an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergent state, and rendering the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground include: Inputting the image of the soybean plant to be detected into the neural radiation field model that has been trained to a convergence state, generating an implicit representation of the soybean plant to be detected, wherein the implicit representation includes three-dimensional information, color, and density distribution of the soybean plant; Based on the implicit representation of the soybean plant to be detected, a two-dimensional image perpendicular to the ground is rendered by projection method; By calculating the volume density of each point at a vertical viewing angle, a density value for each pixel is generated to determine a volume density map at a viewing angle perpendicular to the ground.

6. The soybean plant leaf area index detection method according to claim 1, characterized in that: The step of segmenting the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of soybean leaves in the plant to be detected comprises: Calling a preset image segmentation model, and inputting the two-dimensional image at a viewing angle perpendicular to the ground into the image segmentation model that has been trained to a convergence state; The two-dimensional image at a viewing angle perpendicular to the ground is segmented to output a mask image containing soybean leaves.

7. The soybean plant leaf area index detection method according to claims 1 to 6, characterized in that: The basic network architecture of the image segmentation model is the SAM model.

8. A soybean plant leaf area index detection device, characterized in that: include: An image acquisition module is configured to respond to a soybean plant leaf area index detection instruction and acquire an image of a soybean plant to be detected; a three-dimensional reconstruction module, configured to generate an implicit representation of the soybean plant to be detected according to the soybean plant image to be detected based on the neural radiation field model that has been trained to a convergence state, and render the implicit representation of the soybean plant to be detected to determine a two-dimensional image at a perspective perpendicular to the ground and a volume density map at a perspective perpendicular to the ground; a vertical projection area determination module, configured to determine the area of ​​each pixel in the soybean leaf region, segment the two-dimensional image based on a pre-trained image segmentation model to obtain a mask image of the soybean leaves in the plant to be detected, calculate and determine the number of pixels in the soybean leaf region in the mask image, and determine the vertical projection area of ​​the soybean plant according to the number of pixels in the soybean leaf region and the area of ​​each pixel in the soybean leaf region; a projection area sum determination module, configured to calculate and determine the product between the volume density map at a viewing angle perpendicular to the ground and the mask image to determine the pixel values ​​of all leaves of the soybean plant, and determine the vertical projection area sum of all leaves in an unobstructed state according to the pixel values ​​of all leaves of the soybean plant and the area of ​​each pixel point in the soybean leaf area; The leaf area index detection module is configured to determine the leaf area index of the soybean plant based on the sum of the vertical projection areas of all leaves in the unobstructed state and the vertical projection area of ​​the soybean plant, so as to complete the detection of the leaf area index of the soybean plant.

9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images