A method for three-dimensional imaging of plant chlorophyll fluorescence

Abstract

The application discloses a kind of plant chlorophyll fluorescence three-dimensional imaging method, belong to plant chlorophyll fluorescence three-dimensional field.The plant chlorophyll fluorescence three-dimensional imaging method of present application is complicated, needs multi-angle shooting, synthesis image needs a lot of data and so on for the problems existing in prior art, the present application provides a kind of plant chlorophyll fluorescence three-dimensional imaging device and method, and plant single-view image is obtained under different position light source, and plant is three-dimensionally reconstructed by plant neural radiation field, generates plant fine three-dimensional model, simultaneously, plant fine three-dimensional model is optimized in combination with rendering, obtains plant three-dimensional image and plant chlorophyll fluorescence three-dimensional image, by which the precision and speed of plant chlorophyll fluorescence three-dimensional imaging under single view angle can be improved, data processing time is reduced, more accurate, comprehensive information is provided for plant growth, disease and adversity response and the like research.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional chlorophyll fluorescence in plants, and more specifically, to a method for three-dimensional imaging of chlorophyll fluorescence in plants. Background Technology

[0002] Under natural or agricultural production conditions, chlorophyll content is one of the standards for measuring plant growth. Three-dimensional imaging of plant chlorophyll fluorescence can visually display plant growth traits and monitor plant growth status. Furthermore, using the three-dimensional geometry of plant chlorophyll fluorescence can guide plant cultivation decisions. Therefore, rapid and accurate three-dimensional imaging of plant chlorophyll fluorescence is crucial for plant phenotypic analysis, parameter measurement, and virtual visualization.

[0003] Currently, various 3D imaging representation methods exist, such as voxels, point clouds, and meshes. With advancements in neural radiation fields for multi-view reconstruction and new view synthesis, the accuracy and real-time performance of 3D imaging have significantly improved, providing more feasible methods for 3D reconstruction. While neural radiation fields have achieved some success in multi-view 3D imaging, modeling exploration in single-view scenes remains relatively limited, indicating significant room for development in single-view 3D imaging. Compared to multi-view 3D imaging techniques, the recently emerging neural radiation field, which utilizes illumination and shadow information from a single viewpoint, can achieve single-view 3D imaging, offering significant advantages. First, single-view 3D imaging using neural radiation fields does not require data from multiple views; only a single set of data is needed for 3D reconstruction, resulting in lower data requirements and faster 3D imaging. Second, since it does not require consideration of the correspondence between multiple views, implementation is simpler, reducing complexity. Furthermore, single-view 3D imaging using neural radiation fields offers higher real-time performance, making it more suitable for applications requiring rapid acquisition of plant 3D information.

[0004] Therefore, single-view 3D imaging of neural radiation fields is a more efficient, simple, and real-time 3D imaging technology with broad application prospects. In the plant field, single-view 3D imaging of neural radiation fields can achieve rapid and accurate 3D chlorophyll fluorescence imaging of plants, thus providing strong support for research on 3D imaging, morphological information extraction, quantitative analysis, and virtual reality of crop plants. Summary of the Invention

[0005] 1. Technical problems to be solved

[0006] To address the problems of existing three-dimensional imaging methods for plant chlorophyll fluorescence, such as complexity, the need for multi-angle shooting, and the large amount of data required for image synthesis, this invention provides a device and method for three-dimensional imaging of plant chlorophyll fluorescence. The method acquires single-view images of the plant under different light sources and performs three-dimensional reconstruction of the plant using the plant's neural radiation field to generate a detailed three-dimensional model of the plant. Simultaneously, the detailed three-dimensional model of the plant is optimized by rendering to obtain a three-dimensional image of the plant and a three-dimensional image of plant chlorophyll fluorescence.

[0007] 2. Technical Solution

[0008] The objective of this invention is achieved through the following technical solutions.

[0009] A three-dimensional imaging device for chlorophyll fluorescence of plants includes a dark box and a sealing cover at one end of the dark box. A tray is provided at the other end of the dark box, and the sealing cover and the tray seal the dark box. A light source device is provided inside the dark box near the sealing cover. The light source device includes a white light source and a blue-violet light source. Several white light sources and several blue-violet light sources are arranged alternately around the dark box.

[0010] Furthermore, a multispectral camera is connected to the edge of the sealing cover. The multispectral camera is located inside the dark box, and the angle between the multispectral camera and the vertical plane is 90° to 180°.

[0011] Furthermore, an installation track is provided at one end of the dark box near the sealing cover, and a light shield is installed inside the installation track.

[0012] Furthermore, it also includes a control terminal and a server. One end of the control terminal is connected to the dark box electrical control, and the other end of the control terminal is connected to the server electrical control.

[0013] A three-dimensional imaging method for plant chlorophyll fluorescence includes the following steps:

[0014] Construct a three-dimensional imaging device for plant chlorophyll fluorescence;

[0015] Images of the plant, plant shadows, and plant chlorophyll fluorescence under light sources at different locations were collected. The plant images and shadows were then overlaid to obtain a color image of the plant.

[0016] Extract the shadow region information from the plant image and the plant shadow image, and input the shadow region information from the plant image and the plant shadow image into the neural radiation field for training to obtain the plant neural radiation field.

[0017] A detailed 3D model of the plant is generated by reconstructing the plant's neural radiation field.

[0018] The plant color image and the plant chlorophyll fluorescence image are rendered into a detailed 3D model of the plant to obtain a 3D image of the plant and a 3D image of the plant chlorophyll fluorescence.

[0019] Furthermore, with a white light source enabled, the multispectral camera simultaneously activates the red, green, and blue light bands to capture images of plant shadows; the plant images captured at the same time using the red, green, and blue light bands, along with the plant shadow images, are superimposed to obtain a color image of the plant; with a blue-violet light source enabled, the multispectral camera activates the near-infrared band to capture images of plant chlorophyll fluorescence.

[0020] Furthermore, a texture consistency loss function is used to constrain the shadow region information extracted from the plant shadow image. The texture consistency loss function is expressed as follows:

[0021]

[0022] Among them, L tex (G) represents the texture consistency loss function, which measures the texture similarity between the generated plant shadow surface and the real plant shadow surface, and I represents the real plant shadow image. This indicates the generation of a plant shadow image, where m and n represent the pixel coordinates of the plant shadow image.

[0023] Furthermore, the specific steps for generating a detailed 3D model of the plant include:

[0024] Calculate the visibility of pixels in the plant image to determine whether the pixel is in the shadow region of the plant shadow image;

[0025] By using the calculated pixel visibility of the plant image, the color and light of each pixel in the plant image are obtained;

[0026] By calculating the color and light of each pixel in the plant image through the plant's neural radiation field, the position and surface normal direction of each pixel in the plant image in three-dimensional space can be obtained.

[0027] By taking the position of each pixel in the plant image and the direction of the surface normal as input, a three-dimensional model of the plant is constructed, generating a detailed three-dimensional model of the plant.

[0028] Furthermore, the detailed 3D model of the plant is optimized using a fuzziness loss function, which is expressed as:

[0029]

[0030] Among them, L blur(G) represents the ambiguity loss function, which measures the ambiguity between the generated plant shadow surface and the real plant shadow surface, and h represents the two-dimensional Gaussian kernel function.

[0031] Furthermore, the three-dimensional image of plant chlorophyll fluorescence is optimized using an adversarial loss function, which is expressed as:

[0032]

[0033] Among them, L adv (G,D) denotes the adversarial loss function, which measures the visual similarity between the generated plant shadow image and the real plant shadow image. G represents the generator, D represents the discriminator, E represents the expected loss term, and x... s This represents the plant shadow image, where Pdata represents the actual distribution of plant shadow data, and p z This indicates the noise distribution.

[0034] 3. Beneficial effects

[0035] Compared with the prior art, the advantages of this invention are:

[0036] This invention discloses a plant chlorophyll fluorescence three-dimensional imaging device and method. From a single-view perspective, it generates a detailed three-dimensional model of the plant using the plant's shadow and shadow area information through the plant's neural radiation field. This detailed three-dimensional model allows for better acquisition of chlorophyll fluorescence three-dimensional imaging, facilitating the monitoring of plant physiological characteristics from a spatial three-dimensional perspective. It also improves the accuracy and speed of single-view chlorophyll fluorescence three-dimensional imaging, reduces data processing time, and provides more accurate and comprehensive information for research on plant growth, disease, and stress response, demonstrating strong practicality and wide applicability. Simultaneously, by acquiring plant color images through multi-spectral camera multi-channel image overlay, it can more accurately restore plant color, simplifying the cumbersome operations of image calibration, color balancing, and brightness adjustment required in traditional plant color imaging. Furthermore, by using a texture consistency loss function to constrain the shadow mapped onto the background and the plant's back contour, the generated plant model becomes more realistic, thereby improving the quality of the detailed three-dimensional plant model. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of the three-dimensional imaging device for plant chlorophyll fluorescence according to an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram illustrating the extraction of plant shadows in an embodiment of the present invention;

[0039] Figure 3 This is a flowchart of the method for obtaining three-dimensional distribution information of chlorophyll fluorescence in plants according to an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the framework of the three-dimensional imaging method for chlorophyll fluorescence in plants according to an embodiment of the present invention.

[0041] The labels in the diagram are as follows: 1. Dark box; 2. Sealed cover; 3. Tray; 4. Light source device; 41. White light source; 42. Blue-violet light source; 5. Light shield; 6. Multispectral camera; 7. Control terminal; 8. Server. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0043] Example 1

[0044] like Figure 1 The image shows a three-dimensional imaging device for plant chlorophyll fluorescence provided in this embodiment. The imaging device includes a dark box 1 and a sealing cover 2 at one end of the dark box 1. A tray 3 is located at the other end of the dark box 1, and the sealing cover 2 and tray 3 seal the dark box 1. A light source device 4 is located inside the dark box 1 near the sealing cover 2. The light source device 4 includes a white light source 41 and a blue-violet light source 42, with several white light sources 41 and several blue-violet light sources 42 arranged alternately around the dark box 1. A multispectral camera 6 is connected to the edge of the sealing cover 2, located inside the dark box 1. The adjustment angle of the multispectral camera 6 is between 90° and 180°. A mounting rail is also provided at the end of the dark box 1 near the sealing cover 2, and a light shield 5 is installed within the mounting rail. The imaging device also includes a control terminal 7 and a server 8. One end of the control terminal 7 is electrically connected to the dark box 1, and the other end of the control terminal 7 is electrically connected to the server 8.

[0045] Specifically, in this embodiment, the shape of the dark box 1 includes polygons, cylinders, etc. More preferably, the shape of the dark box 1 is cylindrical. This cylindrical shape ensures that all light sources emitted by the light source device 4 within the dark box 1 maintain a consistent distance from the center point of the plant. Therefore, when acquiring plant shadow images, the shadow changes of the plant will not produce significant deviations, effectively reducing the impact on the three-dimensional imaging of plant chlorophyll fluorescence. The height of the dark box 1 ranges from 30cm to 70cm, allowing for the collection of plants of varying heights. More preferably, the height of the dark box 1 is set to 70cm. In this embodiment, the color of the dark box 1 is black. It should be noted that when the color of the dark box 1 is black, it effectively reduces light reflection because black absorbs light entering the dark box 1, thereby reducing light reflection and preventing light bounce and diffusion inside the dark box 1, thus reducing background interference. Simultaneously, when the color of the dark box 1 is black, it also provides a dark adaptation environment for the plant. It is worth noting that in this embodiment, the material of the dark box 1 is frosted. When the material of the dark chamber 1 is frosted, the influence of external light can be effectively reduced. This is because the frosted material can scatter the external light entering the dark chamber 1, reducing the impact of external light on the three-dimensional imaging of plant chlorophyll fluorescence. This helps maintain the stability of the measurement environment and reduces interference from non-chlorophyll fluorescence signals. Furthermore, when the material of the dark chamber 1 is frosted, it provides a uniform background for the three-dimensional imaging of plant chlorophyll fluorescence, allowing the light emitted by the light source device 4 to be evenly distributed inside the dark chamber 1, avoiding local light intensity differences. This helps obtain accurate chlorophyll fluorescence measurement results and improves measurement repeatability. On the other hand, when the material of the dark chamber 1 is frosted, it effectively enhances the signal-to-noise ratio, reduces light scattering and stray reflection, and increases the signal-to-noise ratio, thereby helping to improve the detection sensitivity and measurement accuracy of chlorophyll fluorescence signals.

[0046] In this embodiment, the sealing cover 2 and tray 3 are also black, and both are made of frosted material, achieving the same effect as the darkroom 1. It should be noted that in this embodiment, the tray 3 comprises upper and lower trays. The upper tray is used to support the plant and can be raised and lowered. Since the height of the multispectral camera 6 is fixed during measurement, if the collected plant is short, the imaging effect of the multispectral camera 6 will be poor. However, by raising and lowering the upper tray, the positional relationship between the plant and the light source device 4 and the multispectral camera 6 can be adjusted, effectively improving the imaging effect of the multispectral camera 6.

[0047] The light source device 4 is used to emit light to the plant. Specifically, the light source device 4 includes a white light source 41 and a blue-violet light source 42, with several white light sources 41 and several blue-violet light sources 42 arranged alternately around the dark box 1. The light source device 4 generates a white LED light source with photochemical light and saturation light of 400nm to 700nm, and a blue-violet LED light source with measurement light of 440nm to 450nm. It should be noted that in this embodiment, the use of white light source 41 for photochemical light and saturation light can avoid measurement errors caused by spectral mismatch, and full-spectrum white light is easier to obtain and more stable than light sources in other wavelength bands. Since the excitation of chlorophyll fluorescence requires irradiation by blue-violet light or red-orange light, and blue-violet light is more effective than red-orange light in exciting fluorescence, blue-violet light has a high absorption efficiency in the wavelength range of the chlorophyll absorption spectrum, which can effectively excite chlorophyll fluorescence emission. At the same time, red-orange light has lower energy and its absorption efficiency is relatively low, which is not conducive to effectively exciting chlorophyll fluorescence. Therefore, the measurement light source 42 is selected as the blue-violet light source. Furthermore, using blue-violet light to excite chlorophyll fluorescence can reduce the influence of background interference. This is because many non-chlorophyll fluorescent components exist in plant tissues, and these components produce strong fluorescence signals under red-orange light, which may mask the measurement results of chlorophyll fluorescence. Choosing a blue-violet light source 42 can reduce interference from non-chlorophyll fluorescence and improve the detection sensitivity of the chlorophyll fluorescence signal. In addition, the blue-violet light source 42 produces a higher signal-to-noise ratio when exciting chlorophyll fluorescence. Therefore, by selecting the blue-violet light source 42, a clearer and stronger chlorophyll fluorescence signal can be obtained, improving the accuracy and reliability of the measurement. In this embodiment, the light intensity of the photochemical light is adjustable in the range of 0–500 μmol·m⁻¹. -2 ·s -1 The intensity of the saturated light is adjustable in the range of 0–4000 μmol·m. -2 ·s -1 .

[0048] In this embodiment, an installation track is provided at one end of the dark box 1 near the sealing cover 2, and a light shield 5 is installed inside the installation track. In this embodiment, the light shield 5 is raised and lowered by the control terminal 7. It should be noted that if a dark environment needs to be created during fluorescence acquisition to avoid the influence of ambient light and to remove background information, the light shield 5 can be lowered; if the environmental background needs to be preserved to obtain the shadow information of the plant shadow image when acquiring plant shadow images, the light shield 5 can be raised.

[0049] Furthermore, a multispectral camera 6 is connected to the edge of the sealing cover 2. The multispectral camera 6 includes a bracket, and is thus connected to the edge of the sealing cover 2 via the bracket. The multispectral camera 6 is located inside the dark box 1. It should be noted that mounting the multispectral camera 6 on the edge of the sealing cover 2 avoids poor image quality due to leaf obstruction when photographing the plant. Additionally, the bracket of the multispectral camera 6 can adjust the angle between the multispectral camera 6 and the vertical plane, which is between 90° and 180°. In this embodiment, more preferably, the angle between the multispectral camera 6 and the vertical plane is 135°. When the angle between the multispectral camera 6 and the vertical plane is 135°, the angle of the plant image is more comprehensive, resulting in more data acquisition. Therefore, in this embodiment, by overlaying multi-channel images from the multispectral camera 6 to obtain a color image of the plant, the plant color can be more accurately reproduced, simplifying the cumbersome operations of image calibration, color balancing, and brightness adjustment required in traditional plant color imaging.

[0050] In this embodiment, the control terminal 7 can control the raising and lowering of the light shield 5 inside the dark box 1, control the opening and closing of the light source device 4, and control the multispectral camera 6 to collect and process data, and display the three-dimensional imaging of plant chlorophyll fluorescence. The server 8 analyzes and processes the plant images, plant shadow images, and plant chlorophyll fluorescence images collected from the multispectral camera 6 under different light sources, reconstructs the three-dimensional image of the plant, and renders it using the fluorescence image.

[0051] Therefore, the plant chlorophyll fluorescence three-dimensional imaging device provided in this embodiment sets a white light source 41 and a blue-violet light source 42 in the dark box 1, and takes pictures of the plant through a multispectral camera 6, thereby realizing the rapid acquisition of plant images, plant shadow images and plant chlorophyll fluorescence imaging, effectively improving the accuracy and speed of plant chlorophyll fluorescence three-dimensional imaging under single view.

[0052] Example 2

[0053] In this embodiment, a plant chlorophyll fluorescence three-dimensional imaging device as described in Example 1 is used to perform chlorophyll fluorescence three-dimensional imaging of the plant. Figure 3 and Figure 4 As shown, the specific steps include:

[0054] A three-dimensional imaging device for plant chlorophyll fluorescence was constructed based on the above-described method. Further, the plant was placed in tray 3, and the angle between the multispectral camera 6 and the vertical plane was adjusted to 135°.

[0055] Furthermore, plant images, plant shadow images, and plant chlorophyll fluorescence images under different light sources were collected. The plant images and plant shadow images were then overlaid to obtain a color image of the plant. Specifically, such as... Figure 2As shown, the control terminal 7 controls the multispectral camera 6 to simultaneously activate the red (R), green (G), and blue (B) light bands, illuminating the plant one by one with the white light source 41 in a clockwise direction. The multispectral camera 6 simultaneously activates the red, green, and blue light bands to capture images of the plant and its shadow under the white light source 41 at different locations. At this time, the wavelength of the red light band is 600nm–700nm, the wavelength of the green light band is 500nm–600nm, and the wavelength of the blue light band is 400nm–500nm. Furthermore, the plant images and shadow images captured simultaneously by the red, green, and blue light bands are superimposed to obtain a color image of the plant, while preserving the plant's background information. Specifically, a color image of the plant is obtained by superimposing the multispectral images corresponding to the plant image and the plant shadow image. It should be noted that the multispectral image refers to the plant image and plant shadow image captured simultaneously by the multispectral camera 6 under a white light source 41 in the red, green, and blue light bands. These images are then superimposed to generate the corresponding color image. Further, the blue-violet light source 42 is turned on, and the multispectral camera 6 is switched to the near-infrared band (N) to capture the chlorophyll fluorescence image of the plant. Specifically, the control terminal 7 controls the lowering of the light shield 5 inside the dark box 1 to allow the plant to undergo dark adaptation for 30 minutes. It should be noted that the purpose of dark adaptation before capturing the chlorophyll fluorescence image is to allow the photosensitive pigments in the plant leaves, such as chlorophyll, to adapt to low-light conditions, ensuring the accuracy and stability of subsequent measurements. After dark adaptation, the photosensitive pigments in the plant leaves can reach a balanced state, avoiding fluorescence signal shifts or changes caused by previous lighting conditions. Based on the chlorophyll fluorescence sensitivity band of the plant, the plant was simultaneously illuminated by a white light source 41 and a blue-violet light source 42, and the current spectral image of the plant in the same scene was acquired by a multispectral camera 6. At this time, the spectral bands of the white light source 41 and the blue-violet light source 42 were 400nm~700nm and 440nm~450nm, respectively. The control terminal 7 controls all white light sources 41 to simultaneously illuminate the plant. The plant is then illuminated with low-intensity light for 20 seconds to allow its chlorophyll to adapt to the light environment, preparing it for fluorescence detection. In this state, the multispectral camera 6 is set to simultaneously activate the red, green, and blue light bands. Then, high-intensity light is projected for 2 seconds under the current light intensity to maximize the fluorescence of the plant's chlorophyll. Finally, the white light sources 41 are turned off, and all blue-violet light sources 42 in the 440nm–450nm band are activated to illuminate the plant. The multispectral camera 6 records the current spectral image of the plant, i.e., the chlorophyll fluorescence image. In this state, the multispectral camera 6 is set to retain only the near-infrared band, and the plant's background information is removed.Therefore, by using the above-mentioned light intensity range, the plant is photographed with a multispectral camera 6 in the light intensity range of 400nm to 700nm to obtain a color image of the plant, and the plant is photographed with a multispectral camera 6 in the light intensity range of 440nm to 450nm to obtain a chlorophyll fluorescence image of the plant.

[0056] Further, shadow region information is extracted from the plant image and the plant shadow image, and this information is input into a Neural Radiance Field (NeRF) for training to obtain the plant neural radiation field. It should be noted that the neural radiation field described in this embodiment is a Neural Reflectance Field from Shading and Shadow under a Single Viewpoint (S3-NeRF). In addition, in this embodiment, for the plant image and the plant shadow image, a ray tracing method is used to determine whether each pixel is in shadow, and the shadow region of the plant shadow image is extracted. The ray tracing method is a prior art technique. In this embodiment, a ray tracing method is used to calculate and determine whether each pixel is in shadow, and to extract the shadow region from the plant shadow image; the ray tracing method calculation formula includes: ray parameterization equation: r = o c +td c , where o c Indicates position 6 of the multispectral camera, d c Let r represent the direction of light rays originating from position 6 of the multispectral camera, r represent a point on the ray, and t represent a parameter. Color calculation equation: C = C0 i *L*(N·L), where C represents the object color calculated after illumination, C i Let L represent the object color, L represent the light intensity, and N represent the normal vector at the intersection point. Furthermore, the neural radiation field is trained using shadow information from plant images and extracted plant shadow images, thereby predicting the radiative transmission of each pixel.

[0057] Furthermore, a detailed 3D model of the plant is generated by reconstructing the plant using its neural radiation field. The specific steps for generating this detailed 3D model include: calculating the pixel visibility of the plant image to determine if the pixel is in the shadow region of the plant's shadow image; using the calculated pixel visibility to obtain the color and light of each pixel in the plant image; calculating the color and light of each pixel in the plant image using the plant's neural radiation field to obtain the position and surface normal direction of each pixel in 3D space; and using the position and surface normal direction of each pixel in the plant image as input to construct the 3D model of the plant, thus generating the detailed 3D model.

[0058] It is worth noting that in this embodiment, a texture consistency loss function is used to constrain the extracted plant shadows from a single viewpoint, thereby obtaining more realistic 3D imaging of plant chlorophyll fluorescence. The texture consistency loss function for the plant shadows is:

[0059]

[0060] Among them, L tex (G) represents the texture consistency loss function, which measures the texture similarity between the generated plant shadow surface and the real plant shadow surface, and I represents the real plant shadow image. This represents the generated plant shadow image, where m and n represent the pixel coordinates of the plant shadow image. Since plant shadows typically consist of occluded and illuminated areas, a texture consistency loss function is used to promote continuity and consistency between the occluded and illuminated areas. Simultaneously, the texture consistency loss function also helps prevent excessive noise and artifacts in the plant shadows.

[0061] It should be noted that the shapes of shadows produced under different lighting conditions vary, thus the light, through the shadows projected onto the background, constrains the outline of the plant's back. In this embodiment, the light visibility of the 3D point is reflected by calculating the occupancy rate between the 3D point and the light source. The formula for calculating light visibility is:

[0062]

[0063] Among them, f v (P l ;x) represents light visibility, P l The position of the white light source 41 is indicated by f, where x represents the position of the specific 3D point used to calculate light visibility. v (P l x)∈[0,1], N LThis represents the number of points sampled by the 3D point on the light source line segment, where i and j represent natural numbers, and x represents the number of points sampled by the light source line segment. i x j This indicates that along the ray r = o c +td c Sampling 3D points.

[0064] It should be noted that calculating the visibility of all sample points along the light ray of a pixel is computationally expensive. Existing technologies generally employ multilayer perceptrons (MLPs) to directly regress the visibility of points or pre-extract surface points after obtaining the scene geometry. In this embodiment, the surface points for root cause localization are located using neural radiation fields to calculate the light visibility of the pixel online. Simultaneously, the color of the pixel is calculated using the following formula:

[0065]

[0066] Where C(r) represents the color of the pixel, f v (P l ;x r (x) represents the visibility of light at a point on a surface. r L represents the position of a surface point located in ray tracing. e Indicates the intensity of the white light source 41 being illuminated, N v f represents the number of samples for each ray. c (x i ,d c ,P l ,L e This indicates that physically based rendering colors occur along the ray r = o. c +td c Sampled 3D point x i The product of d and r. It should be noted that in this embodiment, r represents a point on the ray, used here to determine the position of the pixel, that is, the color of the ray emitted from position 6 of the multispectral camera at the intersection point after ray tracing. c This indicates the direction of light rays originating from position 6 of the multispectral camera, used here to determine the ray direction specified for each pixel. Therefore, this method avoids the additional constraints imposed by multiple viewpoints, reducing the difficulty of three-dimensional imaging of plant chlorophyll.

[0067] In this embodiment, in addition to utilizing the illumination direction information, a fuzziness-based loss function acting on the plant's shadow can be introduced to further optimize the plant's refined 3D model. In this embodiment, the fuzziness loss function is expressed as:

[0068]

[0069] Among them, L blur(G) represents the ambiguity loss function, which measures the ambiguity between the generated plant shadow surface and the real plant shadow surface. h represents the two-dimensional Gaussian kernel function. In the ambiguity loss function applied to plant shadows, the formula for the two-dimensional Gaussian kernel function h is:

[0070]

[0071] Here, σ represents the hyperparameter controlling ambiguity, and p and q represent the distances between pixels. The ambiguity factor of each pixel, i.e., the color difference within its neighborhood, is calculated using a two-dimensional Gaussian kernel function. In the ambiguity loss function, the ambiguity of plant shadows can be used as an indicator to measure the sharpness of shadows in the imaging result. By minimizing the ambiguity of shadows, the sharpness and accuracy of shadow modeling imaging can be further improved, resulting in more accurate three-dimensional imaging results of plant chlorophyll fluorescence.

[0072] Therefore, further, an adversarial loss function is added to the process of plant modeling using shadows in a single-view perspective to further optimize the 3D imaging effect of plant chlorophyll fluorescence. Specifically, in this embodiment, plant shadow information under different lighting conditions is used to improve the accuracy and robustness of plant 3D imaging. In addition to using plant shadow information, an adversarial loss function can also be used to further optimize the 3D imaging effect of plant chlorophyll fluorescence. An adversarial loss function can be used to train a generator network and a discriminator network for adversarial learning. In this embodiment, the adversarial loss function is expressed as follows:

[0073]

[0074] Among them, L adv (G,D) denotes the adversarial loss function, which measures the visual similarity between the generated plant shadow image and the real plant shadow image. G represents the generator, D represents the discriminator, E represents the expected loss term, and x... s This represents the plant shadow image, where Pdata represents the actual distribution of plant shadow data, and p z This represents the noise distribution. In this embodiment, the generator's role is to generate deceptive synthetic shadow images that are difficult for the discriminator to distinguish. Correspondingly, the discriminator's role is to identify the difference between the generated synthetic shadow images and the real shadow images as accurately as possible. Through adversarial learning, the generator network can generate more realistic and lifelike imaging results, thereby obtaining more accurate and reliable three-dimensional imaging results of plant chlorophyll fluorescence.

[0075] Furthermore, in this embodiment, considering the absence of additional constraints from other perspectives under a single viewpoint, adopting a progressively shrinking sampling strategy similar to Unifying Neural Implicit Surfaces and Radiance Fields (UNISURF) would lead to overfitting or underfitting of the model as the sampling interval decreases, thus degrading model performance. Therefore, a strategy combining stereo rendering and surface rendering is adopted. The surface points obtained from root cause localization are used to render colors, and their loss function is calculated. In this embodiment, the loss function for rendering pixel colors based on surface points is expressed as:

[0076] C s (r)=f v (P l ;x r )f c (d c ,P l ,L e ;x r )

[0077] Among them, C s (r) represents the color of the pixel rendered based on the surface point, f c (d c ,P l ,L e ;x r This indicates that the color of a pixel is rendered based on surface points along the ray r = o. c +td c Sampled 3D point x r The product of the two.

[0078] In this embodiment, an occupancy field similar to UNISURF is used to characterize scene geometry. UNISURF uses a multilayer perceptron to map the coordinates of 3D points and the viewing direction to the occupancy value and color of those points, and obtains the color of the pixels through stereo rendering. The loss function based on the color of the pixels in stereo rendering is:

[0079]

[0080] Among them, C l (r) represents the color of the pixel based on stereo rendering, c(x) i ,d c ) represents the color of a 3D point.

[0081] To effectively utilize the shadow information in the photometric stereo image, in this embodiment, the neural radiation field explicitly reconstructs the plant in 3D using the Bidirectional Reflectance Distribution Function (BRDF) of the scene, and uses physically based rendering to obtain the color of the 3D points. Simultaneously, the light visibility of the 3D points in the scene is reconstructed to utilize the rich shadow cues in the image. The final pixel color, i.e., the color of the physically based rendered pixel, is obtained using the following formula, where the loss function for the color of the physically based rendered pixel is:

[0082]

[0083] Among them, C w (r) represents the color of a physically based rendered pixel, f v (P l ;x i ) indicates that light shines on the path along the ray r = o c +td c Sampled 3D point x i The light visibility on the surface, f c (d c ,P l ,L e ;x i This indicates that physically based rendering colors occur along the ray r = o. c +td c Sampled 3D point x i The product of the two.

[0084] In this embodiment, considering non-Lambertian surfaces and spatially varying BRDFs, the 3D point at observation point x is near the point light source (P). l L e Below, observed from the line-of-sight direction, where the line-of-sight direction refers to the vector from the observation point x along the line-of-sight direction toward the multispectral camera 6, the obtained value is expressed as:

[0085] f c (d l ,P l ,L e ;x)=L int (P l ,L e ;x)f m (d l ,ω i (P l ;x);x)max(ω i (P l ;x)·n(x),0)

[0086] Where, dl The x-axis represents the direction of the line of sight at the observation point x, and n represents the direction of the line of sight along the ray r = o. c +td c Sampled 3D point x i The normal vector at point ω i The unit vector representing the direction of light incidence should be noted; here, ω... i This indicates that the near-field point light source P l At this position, observing along the ray r = o from the line of sight. c +td c The incident light direction vector of the sampled 3D point x, which is used to calculate the light intensity, f c (d l ,P l ,L e ;x) represents the BRDF product of the colors of physically rendered pixels, L int (P l ,L e ;x) represents the incident light, ω i (P l ;x) represents the direction of the incident light, f m (d l ,ω i (P l ;x);x) represents the BRDF value of the 3D point at the observation point x.

[0087] Considering the light attenuation issue of point light sources, the illumination intensity at the incident 3D point is calculated based on the distance between the light source and the 3D point. Therefore, diffuse reflection and specular reflection are used to represent the BRDF model, which is expressed as follows:

[0088] f m (ω i ,ω o ;x)=ρ d +ρ s (ω i ,ω o ;x)

[0089] Among them, f m (ω i ,ω o ;x) represents the BRDF model, ω i The unit vector representing the direction of light incidence should be noted; here, ω... i Used to calculate along the ray r = o c +td c The diffuse and specular reflection components at the sampled 3D point x are used to calculate the BRDF value at that point, ω. o ρ is the unit vector representing the direction of light emission. d Represents diffuse color, ρs ρ represents specular reflectivity. d and ρ s The BRDF model is represented by a combination. Further, the specular reflectivity is represented by a weighted combination of Sphere Gaussian coordinates. In this embodiment, the weighted combination of Sphere Gaussian coordinates is expressed as:

[0090]

[0091] Where D(l,n) and G(l,n; λ) represent the weighted combination of the spherical Gaussian function, D(l,n) and G(l,n; λ) are used to describe the influence of the illumination direction and the surface normal on the spherical Gaussian function, l represents the illumination direction, n represents the surface normal, and λ represents the parameter controlling the highlight sharpness.

[0092] Furthermore, the plant color image and the plant chlorophyll fluorescence image are rendered into a detailed 3D model of the plant to obtain a 3D image of the plant and a 3D image of the plant chlorophyll fluorescence. Specifically, the detailed 3D image of the plant is processed pixel by pixel using the plant color image and the plant chlorophyll fluorescence image to obtain the 3D image of the plant and the 3D image of the plant chlorophyll fluorescence.

[0093] Therefore, the plant chlorophyll fluorescence three-dimensional imaging method provided in this embodiment generates a fine three-dimensional model of the plant using the plant's shadow and shadow information and the plant's neural radiation field from a single perspective. By using the generated fine three-dimensional model of the plant, better three-dimensional imaging of plant chlorophyll fluorescence can be obtained. This is not only beneficial for monitoring the physiological characteristics of the plant from a spatial three-dimensional perspective, but also improves the accuracy and speed of single-view plant chlorophyll fluorescence three-dimensional imaging, reduces data processing time, and lowers the difficulty of plant chlorophyll fluorescence three-dimensional imaging. It can provide more accurate and comprehensive information for research on plant growth, disease, and stress response, and has strong practicality and wide applicability.

[0094] The invention and its embodiments have been described above illustratively. This description is not restrictive, and the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. The accompanying drawings are only one embodiment of the invention, and the actual structure is not limited thereto. No reference numerals in the claims should limit the scope of the claims. Therefore, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the invention, such design should fall within the scope of protection of this patent. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Multiple elements stated in the product claims may also be implemented by a single element through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for three-dimensional imaging of plant chlorophyll fluorescence, applied to a three-dimensional imaging device for plant chlorophyll fluorescence; the three-dimensional imaging device for plant chlorophyll fluorescence includes a dark box (1) and a sealing cover (2) disposed at one end of the dark box (1); a tray (3) is disposed at the other end of the dark box (1); the sealing cover (2) and the tray (3) seal the dark box (1); a light source device (4) is disposed inside the dark box (1) near the end of the sealing cover (2); The light source device (4) includes a white light source (41) and a blue-violet light source (42), with a plurality of the white light sources (41) and a plurality of the blue-violet light sources (42) arranged intersectingly around the dark box (1); characterized in that, The three-dimensional imaging method for plant chlorophyll fluorescence includes: Images of the plant, plant shadows, and plant chlorophyll fluorescence under light sources at different locations were collected. The plant images and shadows were then overlaid to obtain a color image of the plant. Extract the shadow region information from the plant image and the plant shadow image, and input the shadow region information from the plant image and the plant shadow image into the neural radiation field for training to obtain the plant neural radiation field. A detailed 3D model of the plant is generated by reconstructing the plant's neural radiation field. The plant color image and the plant chlorophyll fluorescence image are rendered into a detailed 3D model of the plant to obtain a 3D image of the plant and a 3D image of the plant chlorophyll fluorescence.

2. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 1, characterized in that, A multispectral camera (6) is connected to the edge of the sealing cover (2); the multispectral camera (6) is located inside the dark box (1); the angle between the multispectral camera (6) and the vertical plane is 90° to 180°.

3. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 2, characterized in that, An installation track is provided at one end of the dark box (1) near the sealing cover (2); a light shield (5) is installed in the installation track.

4. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 1, characterized in that, The plant chlorophyll fluorescence three-dimensional imaging device also includes a control terminal (7) and a server (8); one end of the control terminal (7) is electrically connected to the dark box (1), and the other end of the control terminal (7) is electrically connected to the server (8).

5. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 1, characterized in that, Turn on the white light source (41), and the multispectral camera (6) simultaneously turns on the red light band, green light band and blue light band to capture plant shadow images; superimpose the plant images captured at the same time by the red light band, green light band and blue light band and the plant shadow images to obtain a color image of the plant; turn on the blue-violet light source (42), and the multispectral camera (6) turns on the near-infrared band to capture plant chlorophyll fluorescence images.

6. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 1, characterized in that, The texture consistency loss function is used to constrain the extraction of shadow region information from the plant shadow image. The texture consistency loss function is expressed as follows: Among them, L tex (G) represents the texture consistency loss function, which measures the texture similarity between the generated plant shadow surface and the real plant shadow surface, and I represents the real plant shadow image. This indicates the generation of a plant shadow image, where m and n represent the pixel coordinates of the plant shadow image.

7. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 1, characterized in that, The specific steps for generating a detailed 3D model of the plant include: Calculate the visibility of pixels in the plant image to determine whether the pixel is in the shadow region of the plant shadow image; By using the calculated pixel visibility of the plant image, the color and light of each pixel in the plant image are obtained; By calculating the color and light of each pixel in the plant image through the plant's neural radiation field, the position and surface normal direction of each pixel in the plant image in three-dimensional space can be obtained. By taking the position of each pixel in the plant image and the direction of the surface normal as input, a three-dimensional model of the plant is constructed, generating a detailed three-dimensional model of the plant.

8. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 7, characterized in that, The fine 3D model of the plant is optimized using a fuzziness loss function, which is expressed as follows: Among them, L blur (G) represents the ambiguity loss function, which measures the ambiguity between the generated plant shadow surface and the real plant shadow surface, and h represents the two-dimensional Gaussian kernel function.

9. The method for three-dimensional imaging of plant chlorophyll fluorescence according to claim 1, characterized in that, The 3D image of plant chlorophyll fluorescence is optimized using an adversarial loss function, which is expressed as follows: Among them, L adv (G,D) denotes the adversarial loss function, which measures the visual similarity between the generated plant shadow image and the real plant shadow image. G represents the generator, D represents the discriminator, E represents the expected loss term, and x... s This represents the plant shadow image, where Pdata represents the actual distribution of plant shadow data, and p z This indicates the noise distribution.

Images