A seedling three-dimensional reconstruction method and system under a dynamic occlusion condition

Abstract

This invention relates to the field of image processing and proposes a method and system for 3D reconstruction of seedlings under dynamic occlusion conditions. By performing distortion correction and time synchronization calibration on multi-view seedling sequence images, the influence of camera equipment is avoided and image synchronization between cameras is ensured. Then, dynamic occlusion detection is performed to accurately identify the occluded areas of seedling leaves. The occluded areas are restored by image weighted fusion, which improves the robustness of occluded area reconstruction. Furthermore, time-aware point cloud fusion reconstruction accurately marks the spatial regions of leaves and makes the unfolded area of ​​leaves conform to the actual scene. This invention improves the quality and accuracy of 3D reconstruction of seedlings.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method and system for three-dimensional reconstruction of seedlings under dynamic occlusion conditions. Background Technology

[0002] With the rapid development of modern agriculture, the growth status of agricultural crops has gradually become the core objective of modern agricultural precision management. As the initial stage of crop growth, the growth status of seedlings directly determines the final yield and quality. The seedling vigor index, as a key physiological indicator for judging the growth status of seedlings, is of great significance for regulating cultivation measures and optimizing resource input by accurately obtaining the seedling phenotypic parameters contained in the seedling vigor index. Therefore, three-dimensional reconstruction of the growth status of agricultural crops has become an important part of modern agricultural precision management.

[0003] In existing technologies, traditional seedling 3D reconstruction technology is often designed based on static shooting environments. However, with the increasing demands of modern agricultural precision management, higher requirements are placed on dealing with the complex situations of seedlings in real field environments. Due to natural factors such as wind, sunlight, soil, and field organisms in real field environments, seedlings experience irregular shaking and shading, making it difficult for existing technologies to accurately capture the leaf morphology and spatial position of seedlings, thus affecting the quality of seedling 3D reconstruction.

[0004] Therefore, how to design a three-dimensional reconstruction method for seedlings that avoids the influence of complex environments in order to improve the reconstruction quality and accuracy has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, the present invention proposes a method and system for 3D reconstruction of seedlings under dynamic occlusion conditions. By performing distortion correction and time synchronization calibration on multi-view seedling sequence images, the influence of camera equipment is avoided and image synchronization between cameras is ensured. Then, dynamic occlusion detection is performed to accurately identify the occlusion area of ​​the seedling leaves. The occlusion area is restored by image weighted fusion, which improves the robustness of occlusion area reconstruction. In addition, time-aware point cloud fusion reconstruction accurately marks the spatial region of the leaves and makes the unfolded area of ​​the leaves conform to the actual scene. The present invention improves the quality and accuracy of 3D reconstruction of seedlings.

[0006] This invention proposes a method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions, comprising:

[0007] Multi-view seedling sequence images are acquired and preprocessed to construct a cross-view time series image set. The preprocessing includes distortion correction and time synchronization calibration.

[0008] Dynamic occlusion detection is performed on the cross-view time series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time series image.

[0009] The occlusion region is restored based on the occlusion mask of the single frame image to obtain an occlusion-complete fused image. The occlusion region restoration process is based on an image weighted fusion algorithm.

[0010] A three-dimensional point cloud model of seedlings is constructed based on the occlusion-complete fusion image. Leaf segmentation and leaf surface reconstruction calculation are performed on the three-dimensional point cloud model of seedlings. The three-dimensional point cloud model of seedlings is based on depth information, and the leaf surface reconstruction calculation is based on surface reconstruction.

[0011] In summary, based on the above-mentioned method for 3D reconstruction of seedlings under dynamic occlusion conditions, distortion correction and time synchronization calibration are performed on multi-view seedling sequence images to avoid the influence of camera equipment and ensure image synchronization across cameras. Dynamic occlusion detection is then performed to accurately identify the occlusion area of ​​seedling leaves. The occlusion area is then restored by image weighted fusion, improving the robustness of occlusion area reconstruction. Furthermore, time-aware point cloud fusion reconstruction accurately marks the spatial region of the leaves, ensuring that the unfolded area of ​​the leaves conforms to the actual scene. This invention improves the quality and accuracy of 3D reconstruction of seedlings. Specifically, multi-view seedling sequence images are acquired and preprocessed to construct a cross-view time-series image set. The preprocessing includes distortion correction and time synchronization calibration to avoid camera equipment interference and ensure image synchronization across cameras. Dynamic occlusion detection is performed on the cross-view time-series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis, and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time-series image, accurately identifying the occlusion area of ​​the seedling leaves. Based on the single-frame image occlusion masks... Occlusion region restoration processing is performed to obtain an occlusion-complete fused image. The occlusion region restoration processing is based on an image weighted fusion algorithm, which improves the robustness of occlusion region reconstruction. A seedling 3D point cloud model is constructed based on the occlusion-complete fused image. Leaf segmentation and leaf surface reconstruction calculation are performed on the seedling 3D point cloud model. The seedling 3D point cloud model is based on depth information, and the leaf surface reconstruction calculation is based on surface reconstruction. Through time-aware point cloud fusion reconstruction, the spatial regions of the leaves are accurately marked, and the unfolded area of ​​the leaves conforms to the actual scene. This invention improves the quality and accuracy of seedling 3D reconstruction.

[0012] Furthermore, the step of acquiring multi-view seedling sequence images and performing preprocessing to construct a cross-view time series image set specifically includes:

[0013] Acquire multi-view seedling sequence images, wherein the multi-view seedling sequence images are multi-view continuous frame images based on time series order;

[0014] Distortion correction is performed on the multi-view seedling sequence images. The specific algorithm for distortion correction is as follows:

[0015] ,

[0016] ,

[0017] in, , Represents the original x and y coordinates of image pixels. , This represents the x and y coordinates of the image after pixel distortion correction. , , Represents the radial distortion coefficient. , Indicates the tangential distortion coefficient. This represents the distance from an image pixel to the optical axis. ;

[0018] The distortion-corrected multi-view seedling sequence images are normalized for brightness using the histogram equalization algorithm, and time synchronization calibration is performed on the multi-view seedling sequence images based on the timestamp information. The multi-view seedling sequence images in a fixed time window are then constructed into a cross-view time series image set.

[0019] Furthermore, the step of performing dynamic occlusion detection on the cross-view time-series image set to obtain multiple single-frame image occlusion masks specifically includes:

[0020] Inter-frame optical flow analysis is performed on a cross-view time-series image set using the PWC-Net optical flow network to obtain pixel-level optical flow fields, and occlusion regions are detected based on abnormal regions in the pixel-level optical flow fields.

[0021] Then, the edges of the cross-view time series images are extracted according to the Canny operator, and the edge difference between consecutive frames in the cross-view time series images is calculated to obtain the edge abrupt change region, and the edge abrupt change region is marked as a suspected occlusion region.

[0022] The cross-view time-series images are reprojected in 3D based on depth information. Depth reprojection consistency is then detected using a disparity consistency algorithm to identify occlusion areas and obtain multiple single-frame image occlusion masks. The disparity consistency algorithm is as follows:

[0023] ,

[0024] in, Indicates the amount of parallax change. , Represents the original x and y coordinates of image pixels. Represents depth information, Indicates time Depth map at any given time Indicates time Time The three-dimensional spatial transformation matrix.

[0025] Furthermore, the step of performing occlusion region recovery processing based on the single-frame image occlusion mask to obtain an occlusion-complete fused image specifically includes:

[0026] The integrity of the occlusion mask in a single frame image is analyzed based on the visibility scoring mechanism. In the integrity analysis, the higher the image visibility score, the more complete the image is considered to be.

[0027] Based on a confidence scoring mechanism, a proximity analysis is performed on the single-frame image occlusion mask of each single image region. In the proximity analysis, the higher the image confidence score, the closer the images are determined to be.

[0028] The occlusion region in the occlusion mask of a single frame image is subjected to weighted fusion processing. The specific algorithm for weighted fusion processing is as follows:

[0029] ,

[0030] in, , Represents the original x and y coordinates of image pixels. This indicates an image that has been occluded and then fused. Indicates visibility score, Indicates the confidence score. This represents a single-frame image occlusion mask. Indicates ordinal number.

[0031] Furthermore, the step of constructing a 3D point cloud model of the seedling based on the occlusion-complete fusion image specifically includes:

[0032] The occlusion-complete fused image is input into the structured light system, which matches the key points in the occlusion-complete fused image and calculates the fundamental matrix and relative pose of the occlusion-complete fused image. According to the parallax triangulation algorithm, the three-dimensional coordinates of each key point are calculated to form a dense point cloud based on all key points.

[0033] Keypoint projection transformation is performed based on depth information and camera intrinsic parameter matrix. The specific algorithm for keypoint projection transformation is as follows:

[0034] ,

[0035] in, , , These represent the x-coordinate, y-coordinate, and vertical coordinates of the keypoints after projection transformation. , This represents the x and y coordinates of key points in a two-dimensional image plane. , Representing the x and y coordinates of the camera's principal point, This represents the depth value of the key point. , This indicates the focal length of the camera along the x and y axes.

[0036] Furthermore, the step of segmenting leaves and performing leaf surface reconstruction calculations on the three-dimensional point cloud model of the seedling specifically includes:

[0037] Outlier removal and point cloud noise smoothing were performed on the 3D point cloud model of the seedlings.

[0038] The 3D point cloud model of the seedling is segmented using a point cloud convolution algorithm to label and extract leaf regions;

[0039] The blade region is fitted with a point cloud based on the radial basis function to obtain the blade surface.

[0040] Perform a spherical rolling simulation on the blade surface to obtain the trajectory of the sphere's center;

[0041] The area is approximated based on the trajectory of the ball's center to obtain the final leaf area and complete the leaf surface reconstruction.

[0042] Furthermore, the step of approximating the area based on the trajectory of the sphere's center specifically includes:

[0043] The trajectory of the sphere's center is triangulated, and the area of ​​the triangles is calculated using the cross product algorithm. The specific algorithm for calculating the area of ​​the triangles using the cross product algorithm is as follows:

[0044] ,

[0045] ,

[0046] in, This represents the area of ​​the unfolded blade corresponding to the triangle. , Vector representation of the sides of adjacent triangles. This represents the final leaf area, and n represents the number of triangles. Indicates ordinal number.

[0047] This invention proposes a seedling three-dimensional reconstruction system under dynamic occlusion conditions, comprising:

[0048] The preprocessing module is used to acquire and preprocess multi-view seedling sequence images to construct a cross-view time series image set. The preprocessing includes distortion correction and time synchronization calibration.

[0049] The dynamic occlusion detection module is used to perform dynamic occlusion detection on the cross-view time series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time series image.

[0050] The occlusion region recovery module is used to perform occlusion region recovery processing based on the occlusion mask of the single frame image to obtain an occlusion completion fused image. The occlusion region recovery processing is based on an image weighted fusion algorithm.

[0051] The reconstruction module is used to construct a three-dimensional point cloud model of seedlings based on the occlusion-complete fusion image, perform leaf segmentation on the three-dimensional point cloud model of seedlings and perform leaf surface reconstruction calculation, wherein the three-dimensional point cloud model of seedlings is based on depth information and the leaf surface reconstruction calculation is based on surface reconstruction.

[0052] The present invention also provides a storage medium that stores one or more programs, which, when executed by a processor, implement the seedling three-dimensional reconstruction method under dynamic occlusion conditions as described above.

[0053] The present invention also provides a computer device, the computer device including a memory and a processor, wherein:

[0054] The memory is used to store computer programs;

[0055] When the processor executes the computer program stored in the memory, it implements the seedling three-dimensional reconstruction method under dynamic occlusion conditions as described above. Attached Figure Description

[0056] Figure 1 This is a flowchart of the seedling three-dimensional reconstruction method under dynamic occlusion conditions proposed in the first embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of the structure of the seedling three-dimensional reconstruction system under dynamic occlusion conditions proposed in the second embodiment of the present invention.

[0058] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0059] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0060] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0062] Please see Figure 1 The diagram shows a flowchart of a three-dimensional reconstruction method for seedlings under dynamic occlusion conditions proposed in the first embodiment of the present invention. This three-dimensional reconstruction method for seedlings under dynamic occlusion conditions includes steps S01 to S04, wherein:

[0063] Step S01: Collect multi-view seedling sequence images and perform preprocessing to construct a cross-view time series image set;

[0064] It should be noted that in this embodiment, the preprocessing includes distortion correction and time synchronization calibration, and multi-view seedling sequence images are acquired. The multi-view seedling sequence images are multi-view continuous frame images based on time series order.

[0065] Distortion correction is performed on the multi-view seedling sequence images. The specific algorithm for distortion correction is as follows:

[0066] ,

[0067] ,

[0068] in, , Represents the original x and y coordinates of image pixels. , This represents the x and y coordinates of the image after pixel distortion correction. , , Represents the radial distortion coefficient. , Indicates the tangential distortion coefficient. This represents the distance from an image pixel to the optical axis. ;

[0069] The distortion-corrected multi-view seedling sequence images are normalized for brightness using the histogram equalization algorithm, and time synchronization calibration is performed on the multi-view seedling sequence images based on the timestamp information. The multi-view seedling sequence images in a fixed time window are then constructed into a cross-view time series image set.

[0070] Step S02: Perform dynamic occlusion detection on the cross-view time series image set to obtain multiple single-frame image occlusion masks;

[0071] It should be noted that in this embodiment, the dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis, and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time series image. Inter-frame optical flow analysis is performed on the cross-view time series image set according to the PWC-Net optical flow network to obtain the pixel-level optical flow field. Occlusion region detection is performed based on the abnormal regions of the pixel-level optical flow field.

[0072] Then, the edges of the cross-view time series images are extracted according to the Canny operator, and the edge difference between consecutive frames in the cross-view time series images is calculated to obtain the edge abrupt change region, and the edge abrupt change region is marked as a suspected occlusion region.

[0073] The cross-view time-series images are reprojected in 3D based on depth information. Depth reprojection consistency is then detected using a disparity consistency algorithm to identify occlusion areas and obtain multiple single-frame image occlusion masks. The disparity consistency algorithm is as follows:

[0074] ,

[0075] in, Indicates the amount of parallax change. , Represents the original x and y coordinates of image pixels. Represents depth information, Indicates time Depth map at any given time Indicates time Time The three-dimensional spatial transformation matrix.

[0076] Step S03: Perform occlusion region restoration processing based on the occlusion mask of a single frame image to obtain an occlusion-complete fused image;

[0077] It should be noted that in this embodiment, the occlusion area restoration process is based on an image weighted fusion algorithm. The integrity of the occlusion mask of a single frame image is analyzed according to the visibility scoring mechanism. In the integrity analysis, the higher the image visibility score, the more complete the image is determined to be.

[0078] Based on a confidence scoring mechanism, a proximity analysis is performed on the single-frame image occlusion mask of each single image region. In the proximity analysis, the higher the image confidence score, the closer the images are determined to be.

[0079] The occlusion region in the occlusion mask of a single frame image is subjected to weighted fusion processing. The specific algorithm for weighted fusion processing is as follows:

[0080] ,

[0081] in, , Represents the original x and y coordinates of image pixels. This indicates an image that has been occluded and then fused. Indicates visibility score, Indicates the confidence score. This represents a single-frame image occlusion mask. Indicates ordinal number.

[0082] Step S04: Construct a 3D point cloud model of the seedling based on the occlusion-complete fusion image, and perform leaf segmentation and leaf surface reconstruction calculation on the 3D point cloud model of the seedling;

[0083] It should be noted that in this embodiment, the seedling 3D point cloud model is based on depth information, the leaf reconstruction calculation is based on surface reconstruction, the occlusion completion fusion image is input into the structured light system, the structured light system matches the key points in the occlusion completion fusion image and calculates the basic matrix and relative pose of the occlusion completion fusion image, and calculates the 3D coordinates of each key point according to the parallax triangulation algorithm, so as to form a dense point cloud based on all key points;

[0084] Keypoint projection transformation is performed based on depth information and camera intrinsic parameter matrix. The specific algorithm for keypoint projection transformation is as follows:

[0085] ,

[0086] in, , , These represent the x-coordinate, y-coordinate, and vertical coordinates of the keypoints after projection transformation. , This represents the x and y coordinates of key points in a two-dimensional image plane. , Representing the x and y coordinates of the camera's principal point, This represents the depth value of the key point. , This indicates the focal length of the camera along the x and y axes;

[0087] Outlier removal and point cloud noise smoothing were performed on the 3D point cloud model of the seedlings.

[0088] The 3D point cloud model of the seedling is segmented using a point cloud convolution algorithm to label and extract leaf regions;

[0089] The blade region is fitted with a point cloud based on the radial basis function to obtain the blade surface.

[0090] Perform a spherical rolling simulation on the blade surface to obtain the trajectory of the sphere's center;

[0091] The area is approximated based on the ball's center trajectory to obtain the final leaf area and complete the leaf surface reconstruction.

[0092] The trajectory of the sphere's center is triangulated, and the area of ​​the triangles is calculated using the cross product algorithm. The specific algorithm for calculating the area of ​​the triangles using the cross product algorithm is as follows:

[0093] ,

[0094] ,

[0095] in, This represents the area of ​​the unfolded blade corresponding to the triangle. , Vector representation of the sides of adjacent triangles. This represents the final leaf area, and n represents the number of triangles. Indicates ordinal number.

[0096] In summary, based on the above-mentioned method for 3D reconstruction of seedlings under dynamic occlusion conditions, distortion correction and time synchronization calibration are performed on multi-view seedling sequence images to avoid the influence of camera equipment and ensure image synchronization across cameras. Dynamic occlusion detection is then performed to accurately identify the occlusion area of ​​seedling leaves. The occlusion area is then restored by image weighted fusion, improving the robustness of occlusion area reconstruction. Furthermore, time-aware point cloud fusion reconstruction accurately marks the spatial region of the leaves, ensuring that the unfolded area of ​​the leaves conforms to the actual scene. This invention improves the quality and accuracy of 3D reconstruction of seedlings. Specifically, multi-view seedling sequence images are acquired and preprocessed to construct a cross-view time-series image set. The preprocessing includes distortion correction and time synchronization calibration to avoid camera equipment interference and ensure image synchronization across cameras. Dynamic occlusion detection is performed on the cross-view time-series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis, and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time-series image, accurately identifying the occlusion area of ​​the seedling leaves. Based on the single-frame image occlusion masks... Occlusion region restoration processing is performed to obtain an occlusion-complete fused image. The occlusion region restoration processing is based on an image weighted fusion algorithm, which improves the robustness of occlusion region reconstruction. A seedling 3D point cloud model is constructed based on the occlusion-complete fused image. Leaf segmentation and leaf surface reconstruction calculation are performed on the seedling 3D point cloud model. The seedling 3D point cloud model is based on depth information, and the leaf surface reconstruction calculation is based on surface reconstruction. Through time-aware point cloud fusion reconstruction, the spatial regions of the leaves are accurately marked, and the unfolded area of ​​the leaves conforms to the actual scene. This invention improves the quality and accuracy of seedling 3D reconstruction.

[0097] Please see Figure 2 The figure shown is a schematic diagram of the seedling three-dimensional reconstruction system under dynamic occlusion conditions proposed in the second embodiment of the present invention. The system includes:

[0098] The preprocessing module 10 is used to acquire and preprocess multi-view seedling sequence images to construct a cross-view time series image set. The preprocessing includes distortion correction and time synchronization calibration.

[0099] The dynamic occlusion detection module 20 is used to perform dynamic occlusion detection on the cross-view time series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time series image.

[0100] The occlusion region recovery module 30 is used to perform occlusion region recovery processing based on the occlusion mask of the single frame image to obtain an occlusion completion fused image. The occlusion region recovery processing is based on an image weighted fusion algorithm.

[0101] The reconstruction module 40 is used to construct a seedling three-dimensional point cloud model based on the occlusion completion fusion image, perform leaf segmentation on the seedling three-dimensional point cloud model and perform leaf surface reconstruction calculation, wherein the seedling three-dimensional point cloud model is based on depth information and the leaf surface reconstruction calculation is based on surface reconstruction.

[0102] The present invention also proposes a computer storage medium storing one or more programs, which, when executed by a processor, implement the above-described method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions.

[0103] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to realize the above-mentioned method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions.

[0104] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain stored, communicated, propagated, or transmitted programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0105] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0106] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0107] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0108] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions, characterized in that, include: Multi-view seedling sequence images are acquired and preprocessed to construct a cross-view time series image set. The preprocessing includes distortion correction and time synchronization calibration. Dynamic occlusion detection is performed on the cross-view time series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time series image. The step of performing dynamic occlusion detection on the cross-view time-series image set to obtain multiple single-frame image occlusion masks specifically includes: Inter-frame optical flow analysis is performed on a cross-view time-series image set using the PWC-Net optical flow network to obtain pixel-level optical flow fields, and occlusion regions are detected based on abnormal regions in the pixel-level optical flow fields. Then, the edges of the cross-view time series images are extracted according to the Canny operator, and the edge difference between consecutive frames in the cross-view time series images is calculated to obtain the edge abrupt change region, and the edge abrupt change region is marked as a suspected occlusion region. The cross-view time-series images are reprojected in 3D based on depth information. Depth reprojection consistency is then detected using a disparity consistency algorithm to identify occlusion areas and obtain multiple single-frame image occlusion masks. The disparity consistency algorithm is as follows: ， in, Indicates the amount of parallax change. , Represents the original x and y coordinates of image pixels. Represents depth information, Indicates time Depth map at any given time Indicates time Time The three-dimensional spatial transformation matrix; The occlusion region is restored based on the occlusion mask of the single frame image to obtain an occlusion-complete fused image. The occlusion region restoration process is based on an image weighted fusion algorithm. The step of performing occlusion region restoration processing based on the occlusion mask of the single-frame image to obtain an occlusion-complete fused image specifically includes: The integrity of the occlusion mask in a single frame image is analyzed based on the visibility scoring mechanism. In the integrity analysis, the higher the image visibility score, the more complete the image is considered to be. Based on a confidence scoring mechanism, a proximity analysis is performed on the single-frame image occlusion mask of each single image region. In the proximity analysis, the higher the image confidence score, the closer the images are determined to be. The occlusion region in the occlusion mask of a single frame image is subjected to weighted fusion processing. The specific algorithm for weighted fusion processing is as follows: ， in, , Represents the original x and y coordinates of image pixels. This indicates an image that has been occluded and then fused. Indicates visibility score, Indicates the confidence score. This represents a single-frame image occlusion mask. Represents ordinal numbers; A three-dimensional point cloud model of seedlings is constructed based on the occlusion-complete fusion image. Leaf segmentation and leaf surface reconstruction calculation are performed on the three-dimensional point cloud model of seedlings. The three-dimensional point cloud model of seedlings is based on depth information, and the leaf surface reconstruction calculation is based on surface reconstruction.

2. The method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions according to claim 1, characterized in that, The step of acquiring multi-view seedling sequence images and preprocessing them to construct a cross-view time series image set specifically includes: Acquire multi-view seedling sequence images, wherein the multi-view seedling sequence images are multi-view continuous frame images based on time series order; Distortion correction is performed on the multi-view seedling sequence images. The specific algorithm for distortion correction is as follows: ， ， in, , Represents the original x and y coordinates of image pixels. , This represents the x and y coordinates of the image after pixel distortion correction. , , Represents the radial distortion coefficient. , Indicates the tangential distortion coefficient. This represents the distance from an image pixel to the optical axis. ; The distortion-corrected multi-view seedling sequence images are normalized for brightness using the histogram equalization algorithm, and time synchronization calibration is performed on the multi-view seedling sequence images based on the timestamp information. The multi-view seedling sequence images in a fixed time window are then constructed into a cross-view time series image set.

3. The method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions according to claim 1, characterized in that, The step of constructing a 3D point cloud model of the seedling based on the occlusion-complete fusion image specifically includes: The occlusion-complete fused image is input into the structured light system, which matches the key points in the occlusion-complete fused image and calculates the fundamental matrix and relative pose of the occlusion-complete fused image. According to the parallax triangulation algorithm, the three-dimensional coordinates of each key point are calculated to form a dense point cloud based on all key points. Keypoint projection transformation is performed based on depth information and camera intrinsic parameter matrix. The specific algorithm for keypoint projection transformation is as follows: ， in, , , These represent the x-coordinate, y-coordinate, and vertical coordinates of the keypoints after projection transformation. , This represents the x and y coordinates of key points in a two-dimensional image plane. , Representing the x and y coordinates of the camera's principal point, This represents the depth value of the key point. , This indicates the focal length of the camera along the x and y axes.

4. The method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions according to claim 1, characterized in that, The steps of segmenting leaves and reconstructing leaf surfaces in the three-dimensional point cloud model of the seedling specifically include: Outlier removal and point cloud noise smoothing were performed on the 3D point cloud model of the seedlings. The 3D point cloud model of the seedling is segmented using a point cloud convolution algorithm to label and extract leaf regions; The blade region is fitted with a point cloud based on the radial basis function to obtain the blade surface. Perform a spherical rolling simulation on the blade surface to obtain the trajectory of the sphere's center; The area is approximated based on the trajectory of the ball's center to obtain the final leaf area and complete the leaf surface reconstruction.

5. The method for three-dimensional reconstruction of seedlings under dynamic occlusion conditions according to claim 4, characterized in that, The step of approximating the area based on the trajectory of the sphere's center specifically includes: The trajectory of the sphere's center is triangulated, and the area of ​​the triangles is calculated using the cross product algorithm. The specific algorithm for calculating the area of ​​the triangles using the cross product algorithm is as follows: ， ， in, This represents the area of ​​the unfolded blade corresponding to the triangle. , Vector representation of the sides of adjacent triangles. This represents the final leaf area, and n represents the number of triangles. Indicates ordinal number.

6. A seedling three-dimensional reconstruction system under dynamic occlusion conditions, characterized in that, A method for implementing three-dimensional reconstruction of seedlings under dynamic occlusion conditions as described in any one of claims 1-5, comprising: The preprocessing module is used to acquire and preprocess multi-view seedling sequence images to construct a cross-view time series image set. The preprocessing includes distortion correction and time synchronization calibration. The dynamic occlusion detection module is used to perform dynamic occlusion detection on the cross-view time series image set to obtain multiple single-frame image occlusion masks. The dynamic occlusion detection is based on inter-frame optical flow analysis, edge residual analysis and depth reprojection consistency detection. Each single-frame image occlusion mask has a unique corresponding cross-view time series image. The step of performing dynamic occlusion detection on the cross-view time-series image set to obtain multiple single-frame image occlusion masks specifically includes: Inter-frame optical flow analysis is performed on a cross-view time-series image set using the PWC-Net optical flow network to obtain pixel-level optical flow fields, and occlusion regions are detected based on abnormal regions in the pixel-level optical flow fields. Then, the edges of the cross-view time series images are extracted according to the Canny operator, and the edge difference between consecutive frames in the cross-view time series images is calculated to obtain the edge abrupt change region, and the edge abrupt change region is marked as a suspected occlusion region. The cross-view time-series images are reprojected in 3D based on depth information. Depth reprojection consistency is then detected using a disparity consistency algorithm to identify occlusion areas and obtain multiple single-frame image occlusion masks. The disparity consistency algorithm is as follows: ， in, Indicates the amount of parallax change. , Represents the original x and y coordinates of image pixels. Represents depth information, Indicates time Depth map at any given time Indicates time Time The three-dimensional spatial transformation matrix; The occlusion region recovery module is used to perform occlusion region recovery processing based on the occlusion mask of the single frame image to obtain an occlusion completion fused image. The occlusion region recovery processing is based on an image weighted fusion algorithm. The step of performing occlusion region restoration processing based on the occlusion mask of the single-frame image to obtain an occlusion-complete fused image specifically includes: The integrity of the occlusion mask in a single frame image is analyzed based on the visibility scoring mechanism. In the integrity analysis, the higher the image visibility score, the more complete the image is considered to be. Based on a confidence scoring mechanism, a proximity analysis is performed on the single-frame image occlusion mask of each single image region. In the proximity analysis, the higher the image confidence score, the closer the images are determined to be. The occlusion region in the occlusion mask of a single frame image is subjected to weighted fusion processing. The specific algorithm for weighted fusion processing is as follows: ， in, , Represents the original x and y coordinates of image pixels. This indicates an image that has been occluded and then fused. Indicates visibility score, Indicates the confidence score. This represents a single-frame image occlusion mask. Represents ordinal numbers; The reconstruction module is used to construct a three-dimensional point cloud model of seedlings based on the occlusion-complete fusion image, perform leaf segmentation on the three-dimensional point cloud model of seedlings and perform leaf surface reconstruction calculation, wherein the three-dimensional point cloud model of seedlings is based on depth information and the leaf surface reconstruction calculation is based on surface reconstruction.

7. A storage medium, characterized in that, The storage medium stores one or more programs that, when executed by a processor, implement the seedling three-dimensional reconstruction method under dynamic occlusion conditions as described in any one of claims 1-5.

8. A computer device, characterized in that, The computer device includes a memory and a processor, wherein: The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the seedling three-dimensional reconstruction method under dynamic occlusion conditions as described in any one of claims 1-5.

Images