Active lighting and probabilistic occupancy in seedling 3D reconstruction: methods, equipment, and media
By employing active lighting and probabilistic occupancy methods, and utilizing lighting geometric constraints and polarization component decoupling, the problem of 3D reconstruction under static occlusion and weak texture of seedlings was solved, achieving high-precision restoration of internal structure and preservation of phenotypic features.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- EAST CHINA JIAOTONG UNIVERSITY
- Filing Date
- 2026-03-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies are ineffective in 3D reconstruction when dealing with static, weakly textured, and severely self-occluded seedlings, and cannot restore their internal structure.
By actively changing the lighting angle and utilizing the geometric changes in shadows and illuminated areas, the lighting information is transformed into geometric constraints, and a three-dimensional reconstruction method with multi-source lighting constraints is constructed. This method includes a voxel mesh probability occupancy model and lighting epipolar geometric constraints. Combined with polarization component decoupling and dynamic topology restoration mechanisms, the reverse calculation from 2D lighting sequences to 3D depth entities is achieved.
It overcomes the challenge of static dead zone reconstruction, solves the problem of phenotypic measurement in windless environments, eliminates specular highlight interference, accurately preserves minute void features, prevents the loss of phenotypic features in traditional reconstruction, and achieves robust reconstruction under complex lighting conditions.
Smart Images
Figure CN121921463B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the interdisciplinary field of computer vision and smart agriculture, and in particular to a method, device and medium for three-dimensional reconstruction of seedlings based on active lighting and probabilistic occupancy. Background Technology
[0002] Currently, plant 3D reconstruction mainly relies on multi-view stereo vision (MVS) or structured light technology. However, these technologies face two major challenges in agricultural phenotyping scenarios: First, weak textures fail, as seedling leaves have smooth surfaces and highly uniform colors, lacking texture details for feature matching, resulting in voids in leaf areas by traditional MVS algorithms; second, static occlusion blind spots occur, in windless environments such as greenhouses and artificial climate chambers, or for seedlings with rigid stems and tightly packed leaves, the leaves are in a relatively static state, and existing technologies cannot detect the occluded internal structures.
[0003] Therefore, the present invention aims to solve the technical problem of reverse calculation from "2D light and shadow sequence" to "3D depth entity" by actively changing the lighting angle and utilizing the geometric changes of shadow and illuminated area under the constraints of no object movement and no camera movement. Summary of the Invention
[0004] The purpose of this invention is to provide a method, device and medium for three-dimensional reconstruction of seedlings based on active lighting and probabilistic occupancy, so as to solve the problem that the existing technology has poor three-dimensional reconstruction effect and cannot restore the internal structure when dealing with static, weak texture and severely self-occluded seedlings.
[0005] In a first aspect, the present invention provides a static seedling three-dimensional reconstruction method based on multi-source light and shadow constraints, comprising the following steps:
[0006] Using a single fixed image acquisition unit, a set of images containing a background image and multiple active illumination images are simultaneously acquired for a stationary target seedling under the condition that multiple light sources with pre-calibrated spatial coordinates on a circular array are turned on independently in sequence; and a three-dimensional voxel grid surrounding the target seedling is constructed, and an initial occupancy probability value is assigned to each voxel in the three-dimensional voxel grid.
[0007] For any active illumination map and its corresponding active light source, identify the illuminated pixels in the active illumination map; based on the optical center of the image acquisition unit, the illuminated pixels, and the calibrated spatial coordinates of the active light source, construct active light and shadow epipolar geometric constraints; by calculating the spatial geometric intersection of the observation line of the image acquisition unit and the projected light ray of the light source, locate the candidate three-dimensional coordinates of the surface; and update the occupancy probability of voxels in the three-dimensional voxel mesh according to the candidate three-dimensional coordinates of the surface.
[0008] After traversing all the active lighting maps, a probability occupancy field updated with multi-source lighting information is obtained; based on the probability occupancy field, the voxel set belonging to the seedling entity is determined, and the three-dimensional geometric model of the target seedling is generated using the voxel set.
[0009] As an optional implementation of the first aspect of this application, before the step of updating the occupancy probability of voxels in the three-dimensional voxel mesh based on the candidate three-dimensional coordinates of the surface, the method further includes photometric attribute calculation based on polarization component decoupling: extracting vegetation mask based on the color and polarization information of the active illumination map; and calculating the decoupled diffuse reflection difference map using the following formula with a preset orthogonal polarization layout: ;in, For pixels, For the first i Original observed brightness when each light source is turned on. For ambient background brightness, The angle between the camera polarizer and the polarization direction of the light source is denoted as . The polarization component decoupling coefficient is used; based on the diffuse reflection difference map, the illumination state of a pixel is determined as either high-confidence illumination or pure shadow, and the illumination confidence level is calculated. The light-receiving confidence level The occupancy probability is used to update the voxel in the probability occupancy field update step.
[0010] As an optional implementation of the first aspect of this application, the step of updating the occupancy probability of voxels in the three-dimensional voxel mesh based on the candidate three-dimensional coordinates of the surface specifically includes: for the pixels with high confidence of receiving light, calculating the spatial geometric intersection of the camera's line of sight and the light ray projected by the light source. The basic visibility confidence factor acting on voxels in the three-dimensional voxel mesh is calculated using the following formula: ;in, voxels Center to the spatial geometric intersection point The geometric distance, σ is the physical adjustment parameter for compensating for the penumbra at the leaf edge; the basic visibility confidence factor is used to measure the voxel. Update the occupancy probability and output the updated occupancy probability.
[0011] As an optional implementation of the first aspect of this application, the basic visibility confidence factor is utilized. For voxels Before performing the step of updating the occupancy probability, it also includes a void topology protection correction based on a dynamic confidence gate: for voids located at the intersection of the light source and the spatial geometry. Obtain the light-receiving confidence of voxels along the path. Set a threshold for strong factual evidence. ;like The voxel was determined to be in a state of strong light path penetration, and the final confidence level was directly adjusted. Equal to the aforementioned basic visibility confidence factor To force the preservation of physical voids; if Then, the sum of the occupancy probabilities of all voxels in the spatial neighborhood of the given voxel is calculated. The final confidence level is calculated using the following formula. : ;in, This is a numerical truncation function used to truncate numerical values. Limited to between 0 and 1; Weighting coefficients used to adjust the intensity of the game; To prevent extremely small values where the denominator is zero; through the final confidence level. Update the voxels The probability of occupancy.
[0012] As an optional implementation of the first aspect of this application, a revised final confidence level is generated. Used to update the voxel The occupancy probability is iteratively updated using a nonlinear decay criterion: ;in, and These represent the voxel occupancy probabilities before and after the update; the iterative update achieves accurate preservation of real physical voids and adaptive softening protection of low-confidence blade edge regions.
[0013] As an optional implementation of the first aspect of this application, before the step of determining the set of voxels belonging to the seedling entity based on the probability occupancy field, a shadow consistency check is further included: for voxels whose occupancy probability is higher than a preset threshold after the initial update. Through virtual projection function Simulate its light source in each of the aforementioned calibrated spatial coordinates The theoretical lighting state is determined; the theoretical lighting state is compared with the actual lighting state to calculate the voxel shadow consistency score. If the shade consistency score (Score) The shading consistency score indicates that the voxel has consistent optical path transmission characteristics under multi-source observation. For the voxels The occupancy probability is subject to a mandatory second-weighted correction to remove artifacts left by the neighborhood protection mechanism and output the final probability occupancy field.
[0014] As an optional implementation of the first aspect of this application, the step of determining the voxel set belonging to the seedling entity based on the final probability occupancy field and generating a three-dimensional geometric model of the target seedling using the voxel set specifically includes: setting a global determination threshold. If the occupancy probability of a voxel after double weighting is higher than If the voxel is determined to be a leaf entity, the moving cube algorithm is used to extract the isosurface of the final probability occupancy field to generate an initial triangular mesh describing the outer contour of the seedling. The initial triangular mesh is then subjected to Laplacian smoothing and morphological filtering to eliminate the quantization step effect and remove isolated floating fragments, and finally output the optimized three-dimensional geometric model.
[0015] As an optional implementation of the first aspect of this application, the method further includes: the image acquisition unit is a global shutter industrial camera with a polarizer, fixed vertically downward; the multiple light sources on the ring array are high-brightness LED point light sources with polarizers, and their polarization directions are orthogonal to the polarizer direction of the camera; the light source with pre-calibrated spatial coordinates is the three-dimensional coordinates of all light sources in the world coordinate system obtained by precise calibration, and the three-dimensional coordinates are the geometric reference for constructing the active light and shadow epipolar geometric constraint.
[0016] In a second aspect, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.
[0017] Thirdly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.
[0018] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0019] 1. Overcoming the challenge of static blind spot reconstruction: Complex layered structures can be detected simply by changing the active annular light field, without the need for plant or camera movement, thus solving the problem of phenotypic measurement in windless greenhouse environments.
[0020] 2. Physical layer suppression of strong reflection interference: Through the polarization component decoupling mechanism, the interference of specular highlights on the cuticle of seedling leaves on geometric reconstruction is eliminated from the physical source, ensuring robustness under complex lighting conditions.
[0021] 3. Dynamic topology restoration capability with physical perception: Introducing a game mechanism of "optical path factual evidence" and "neighborhood space priors." The algorithm can intelligently distinguish between "real biological voids, such as insect-eaten holes and natural cracks" and "false edge noise," such as penumbra. While ensuring smooth and continuous leaf edges, it accurately preserves minute internal void features, effectively preventing the loss of phenotypic features caused by "over-repairing" in traditional 3D reconstruction.
[0022] 4. Reverse reasoning that turns disadvantages into advantages: Unlike the traditional MVS's reliance on texture, this invention utilizes the physical exclusivity of the impermeability of light paths to transform light and shadow information into high-intensity geometric constraints, filling the reconstruction gaps in areas with weak texture and homogeneous blades. Attached Figure Description
[0023] Figure 1 This is a schematic diagram of the hardware system in an embodiment of the present invention;
[0024] Figure 2 This is an algorithm flowchart of a static seedling three-dimensional reconstruction method based on multi-source light and shadow constraints in an embodiment of the present invention;
[0025] Figure 3 This is a schematic diagram illustrating the principle of active light and shadow epipolar geometry constraint in an embodiment of the present invention;
[0026] Figure 4 This is a schematic diagram illustrating the principle of the void topology protection and correction mechanism in this embodiment of the invention. Detailed Implementation
[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0028] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0029] This invention provides a static seedling 3D reconstruction method based on multi-source light and shadow constraints, which is implemented based on a specific hardware system. The specific implementation process of this method will be described in detail below from the aspects of system hardware composition and algorithm flow.
[0030] I. System Hardware Components (e.g.) Figure 1 (As shown)
[0031] like Figure 1 As shown, the hardware system of this embodiment of the invention is built in a dark room or a well-lit environment to reduce interference from ambient stray light. The system mainly includes an image acquisition unit, an active illumination unit (ring light source array), a synchronization control unit, and a computer.
[0032] The image acquisition unit includes a CMOS global shutter industrial camera with a resolution of [resolution missing]. 10,000 pixels (e.g.) It is fixedly installed vertically downwards. Its optical axis is directly aligned with the plane where the seedling is placed. The lens is a low-distortion fixed-focus lens (focal length...). ), optical axis distance to seedling placement plane height Approximately A polarizing filter is installed in front of the lens, which, together with the light source polarizing filter, eliminates specular reflection from the cuticle of the leaf blades.
[0033] The active lighting unit is a ring-shaped light source array, consisting of... A high-brightness LED point light source (in this embodiment, we take...) ), evenly distributed on a circle centered on the camera's optical axis (radius) The incident angle of the light source is approximately Ensure the shadow length is appropriate and illuminates the lower leaves. Spatial coordinates of all light sources. All have been precisely calibrated beforehand. The coordinates of the light source obtained through calibration... It is the sole geometric reference for the subsequent construction of the active polar geometry plane.
[0034] The synchronization control unit uses an STM32 or FPGA to achieve microsecond-level synchronization control. Its workflow is as follows: pull high the first... One light source pin illuminates the LED; delay Wait for the light source brightness to stabilize; send a trigger signal to the camera to trigger its exposure (e.g., exposure time). After the exposure is complete, turn off the current light source, delay for a short time, and then turn on the next light source. Repeat this process until all light sources have been cycled through.
[0035] II. Algorithm Flow (e.g.) Figure 2 (As shown)
[0036] Step 1: Acquisition of multi-source light and shadow sequences.
[0037] The synchronization control unit is activated, and N LEDs in the ring light source array are illuminated sequentially according to a preset timing sequence. During the independent illumination of each light source, the image acquisition unit is synchronously triggered to expose and capture one frame of image. First, a background image is acquired with all light sources off. Then, N active illumination images were collected sequentially. The final result is an image sequence containing one background image and N active lighting images.
[0038] Step 2: Calculation of photometric properties based on polarization component decoupling.
[0039] To address the strong specular reflection (highlights) generated by the cuticle of seedling leaves and the interference from the complex ambient light in the greenhouse, this step achieves the separation of diffuse reflection components and specular reflection interference by actively controlling the polarization state and image difference.
[0040] First, vegetation mask extraction. For each active illumination image... Vegetation segmentation is performed using color information. The image is converted from RGB to Lab color space. Since plant leaves have significant negative values in the a channel (green-red component), a preliminary vegetation mask can be extracted by setting a negative threshold. To further exclude non-vegetation areas, the mask is weighted and corrected based on polarization characteristics to ensure segmentation accuracy.
[0041] Next, polarization component decoupling and diffuse reflection extraction are performed. Using the pre-set orthogonal polarization layout in the system hardware, the decoupled diffuse reflection difference map is calculated using the following formula. This image excludes the specular reflection component, which carries interfering information, and retains only the diffuse reflection information, which carries the geometric information of the blades.
[0042] ;
[0043] in, For pixels, For the first i Original observed brightness when each light source is turned on. For ambient background brightness, The relative angle between the camera polarizer and the polarization direction of the light source (in this embodiment, it is set to...). To achieve orthogonal extinction). These are the decoupling coefficients for the polarization components, used to quantify and compensate for energy loss during optical transmission. The diffuse reflection difference map calculated in this step has physically eliminated most of the influence of specular reflection highlights.
[0044] Finally, a pure light-receiving state is determined. A high-confidence light-receiving threshold is set. and a pure shadow threshold ,for Each pixel in:
[0045] like This indicates that the point is directly illuminated by the light source, and since specular interference has been eliminated, its brightness truly reflects the diffuse reflection characteristics of the surface. The calculated confidence level of illumination is... ;
[0046] like and This indicates that the point is in the umbra or penumbra of the active light source.
[0047] This step provides more physically reliable two-dimensional constraint information for subsequent 3D reconstruction. Specifically, unlike the traditional shadowing method which is easily affected by environmental interference, the method of this invention uses polarization decoupling coefficients... The high-gloss "false features" of the leaf cuticle were eliminated at the acquisition front end. This mechanism of "physical noise reduction + mathematical decoupling" ensures that the "light-receiving state" on which the reverse light path removal is based in the subsequent steps has extremely high physical accuracy, thereby realizing sub-pixel level extraction of the occlusion boundary.
[0048] Step 3: Spatial sculpting based on the geometric constraints of light and shadow and the probabilistic occupancy model.
[0049] This step utilizes the diffuse reflection difference map extracted in step two, which has eliminated specular reflection (highlight) interference. As the input signal, by constructing an active epipolar geometric constraint of "virtual camera - real light source" and introducing a probabilistic occupancy model, a technological evolution from traditional "air removal" to "precise surface positioning" is achieved. Through soft updates of the physical optical path characteristics, the reconstruction challenges of plant leaf edges, micro-cavities, and complex shading areas are effectively solved.
[0050] First, voxel space probabilistic initialization is performed. A target 3D voxel mesh set surrounding the seedling is constructed. To characterize the uncertainty of the initial state and support probabilistic fusion of multi-source lighting and shadows, for each voxel... Assign initial occupancy probability value (in ), indicating that the initial state is unknown.
[0051] Secondly, the active lighting and shadow polar geometry system is constructed. This involves calibrating each active LED light source. Treated as a "virtual projection center," it is used in conjunction with a real monocular camera. Pre-defined spatial baselines between them are used to construct active light and shadow epipolar geometric constraints (such as...). Figure 3 (As shown). Specifically, establishing a light source With camera optical center The three-dimensional baseline vector between A virtual binocular baseline serving as the spatial baseline; for pixels on the image plane determined to be illuminated. The corresponding camera observation line of view With light source Emitted projected light Together they form an active polar plane in three-dimensional space.
[0052] Subsequently, physical beam modeling and precise surface point localization are performed. Unlike traditional spatial sculpting, which only removes air within the view frustum, this method precisely searches the surface by solving for the geometric relationships of spatial rays. The spatial geometric intersection of the camera's line of sight and the light source's rays is calculated. The candidate 3D coordinates of the edge of the obstructed blade are directly obtained, and the geometric intersection point is represented as follows: ;
[0053] Considering the energy attenuation of the point source and the diffraction effect at the blade edge, the confidence distribution of the beam on the spatial voxel is defined, and the fundamental visibility confidence factor is calculated. :
[0054] ;
[0055] in, voxels Center to spatial geometric intersection The geometric distance is σ, which is the physical adjustment parameter for compensating for the penumbra at the blade edge. This confidence factor is determined by the diffuse reflection intensity after polarization decoupling, ensuring that the probability update is not misled by the cuticle highlights on the blade surface.
[0056] Furthermore, an adaptive update of the reverse occupancy probability is performed.
[0057] To address the issue of traditional algorithms easily misfilling real voids or excessively corroding blade edges, this embodiment introduces a dynamic game mechanism combining physical optical path evidence and spatial neighborhood priors. This mechanism incorporates the basic confidence factor... Corrected to final removal weight Specifically, it includes the following logical decision-making process: (e.g.) Figure 4 (as shown)
[0058] The first step is to determine strong factual evidence. This involves setting a strong factual threshold. (In this embodiment, we take) For any voxel along the optical path, check its corresponding diffuse reflection differential light reception confidence level. If the current illuminated pixel's illumination confidence level This indicates that the light path penetration signal is extremely strong, confirming that the region is a physically existing biological cavity. At this point, the weighted areas are finally removed. It forces the ignoring of neighborhood protection and performs high-intensity spatial carving to ensure that tiny insect holes are not mistakenly filled.
[0059] The second step is to implement topology protection based on signal-to-noise ratio game theory. Only when... The hole topology preservation mechanism is activated only when the optical path signal is at a low to medium confidence level, possibly due to leaf edge diffraction or a penumbra. At this time, the algorithm calculates voxels. neighborhood Inside (e.g.) Average occupancy probability of neighborhood voxels The corrected elimination weights are calculated using the following competitive game formula:
[0060]
[0061] in, This is a numerical truncation function used to restrict the calculation results to the interval [0, 1]. To adjust the weighting coefficients of the game intensity (in this embodiment, we take...) ); To prevent the minimum value where the denominator is zero (e.g.) ).
[0062] The physical meaning of this formula lies in establishing a competitive suppression model based on signal-to-noise ratio: the denominator... The strength of physical evidence for light path penetration events was quantified as an active removal driving force that promotes voxel voiding; while molecular... The spatial neighborhood constraints of voxels supported by surrounding entities are quantified, serving as topological inertia to maintain their current occupied state.
[0063] When the strength of the physical evidence of light path penetration increases significantly (i.e., the denominator increases), this driving force can overcome the topological inertia of the neighborhood, reducing the suppression term and thus making... Maintain high values and prioritize removal based on physical facts;
[0064] When the optical path evidence is weak and subject to strong neighborhood constraints (i.e., the numerator is large and the denominator is small), topological preserving inertia dominates, causing the suppression term to approach 1, thus making... When the optical path confidence level approaches zero, protection based on geometric priors is prioritized. This mechanism constructs a competitive relationship between physical evidence and neighborhood priors by using the optical path confidence level as the denominator: ensuring that when the optical path evidence is weak and the probability of neighborhood occupancy is high, the elimination weight is reduced to prevent edge fragmentation; while when the optical path evidence is conclusive, the neighborhood inhibition effect is significantly weakened, prioritizing the preservation of true holes. Finally, the calculated final confidence level is used... Update the voxels The probability of occupancy.
[0065] Using the revised The voxel state is updated according to the nonlinear decay criterion, and the occupancy probability iterative update can be expressed as:
[0066] ;
[0067] This achieves high-confidence physical removal (precisely reconstructing air regions). ) and low-confidence edge softening protection (blade boundary) Adaptive balancing (slow updates).
[0068] Finally, multi-source cross-validation and determination are performed. This is done after completing the circular array. After polling and probing the sequence of light sources, the generated probability field is subjected to multi-source consistency verification and binarization segmentation to ensure the integrity and accuracy of the reconstruction results.
[0069] For the initially preserved voxels Calculate its shadow consistency score Using virtual projection functions Simulate the theoretical illumination state of voxels under various calibrated light sources and compare it with the obtained actual illumination state. Perform a comparison; the formula for shadow consistency verification is expressed as follows:
[0070]
[0071] This embodiment specifically introduces a secondary forced correction mechanism: if a voxel is temporarily retained (determined to be an entity) in the aforementioned neighborhood protection step, but is found to be an entity in this step... If the light penetration is extremely low (i.e., it appears as light penetration at most lighting angles), then the previous protection was determined to be a false alarm. The algorithm will then base its judgment on... The occupancy probability of the voxel is forcibly decayed to eliminate artifacts left by the neighborhood protection mechanism.
[0072] Set global judgment threshold If voxels satisfy If the consistency score is higher than the preset threshold, the voxel is ultimately determined to belong to the leaf entity; otherwise, it is determined to be a background air region. A high-precision 3D structure is sculpted through closed-loop verification with multi-angle lighting constraints. Step 4: Surface reconstruction and meshing output.
[0073] To transform a discrete probability occupancy field into a continuous geometric entity that can be used for phenotypic analysis, the following topology reconstruction steps are performed:
[0074] (1) Isosurface extraction: The Marching Cubes algorithm is used to extract the surface of the processed voxel space. The vertex positions of the triangular facets are determined according to the occupancy probability gradient of each voxel, and a continuous mesh describing the outer contour of the seedling is generated.
[0075] (2) Geometric optimization and smoothing: The original generated mesh is smoothed using Laplacian to eliminate the quantization step effect during the digitization process. At the same time, morphological operators are used to remove small, isolated, suspended debris to ensure the topological continuity of the blade surface.
[0076] (3) Results output and analysis: The reconstructed seedling geometric entity is exported as a standard 3D model format (such as .OBJ or .PLY). This model fully preserves the leaf layering relationship and deep geometric information of the shading area of the seedling, which serves as the basis for subsequent quantitative analysis of phenotypic parameters such as leaf area, plant height and growth rate.
[0077] In summary, it should be noted that the number of light sources The number of sensors is not limited to 8 and can be adjusted according to accuracy requirements; the light source type can be replaced with a near-infrared light source to work with specific sensors; the voxel mesh can be replaced with an octree structure to accelerate computation; this solution can be used as an enhancement module for binocular stereo vision, using binoculars to obtain basic depth and using the active lighting and shadow of this solution to repair holes in occluded areas.
[0078] This invention breaks through the simple binary causal logic of traditional spatial sculpting and establishes a "pixel-based light reception" system based on the physical properties of light and shadow. Active epipolar geometry positioning Probabilistic optical path reachability A novel causal reasoning chain of "topology adaptive occupation update".
[0079] Its key feature, which distinguishes it from traditional MVS (Multi-view Stereo Vision) algorithms, is:
[0080] 1. Shift from "feature matching" to "physical constraints":
[0081] Traditional MVS relies heavily on the texture details of leaf surfaces for feature matching, which easily leads to voids or matching failures in areas like seedling leaves where color is highly homogeneous and texture is weak. This invention utilizes a pre-calibrated active light source to construct a "virtual binocular" baseline. By calculating the geometric intersection of the camera's line of sight and the active light path, it achieves texture-independent and accurate geometric reconstruction, forcibly establishing geometric constraints in textureless areas.
[0082] 2. Shifting from "binary removal" to "probabilistic sculpting":
[0083] Unlike traditional algorithms that are sensitive to noise and operate on a "black and white" logic, this invention introduces a voxel occupancy probability model. This is achieved through a basic visibility confidence factor. Simulating beam energy attenuation and diffraction effects gives the reconstruction process strong noise resistance and fault tolerance, enabling it to more realistically reproduce sub-pixel-level geometric details of the blade edges.
[0084] 3. Possesses adaptive capabilities based on biological topology awareness:
[0085] To address the unique complexities of plant phenotypes (such as insect-eaten cavities and overlapping leaves), this invention innovatively proposes a neighborhood feedback regulation mechanism. By recognizing the supporting role of local occupancy probabilities, the algorithm can distinguish between "pure background air" and "biological cavities," effectively preventing the "over-sculpting" and edge topological tearing problems commonly encountered by traditional sculpting algorithms when dealing with complex structures.
[0086] Optionally, this application embodiment also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described embodiment of the active lighting and probability occupancy seedling three-dimensional reconstruction method and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0087] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of the active lighting and probability occupancy seedling three-dimensional reconstruction method and achieve the same technical effect. To avoid repetition, they will not be described again here.
[0088] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0089] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0090] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0091] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. A method for three-dimensional reconstruction of seedling from active light and shadow and probability occupation, characterized in that, Includes the following steps: Using a single fixed image acquisition unit, a set of images containing a background image and N active illumination images are simultaneously acquired for a stationary target seedling under the condition that N light sources with pre-calibrated spatial coordinates on a circular array are turned on independently in sequence. And construct a three-dimensional voxel grid surrounding the target seedling, and assign an initial occupancy probability value to each voxel in the three-dimensional voxel grid; For any of the active illumination maps and their corresponding active light sources, identify the illuminated pixels in the active illumination map; Based on the optical center of the image acquisition unit, the illuminated pixel, and the calibrated spatial coordinates of the activated light source, an active light and shadow epipolar geometric constraint is constructed. By calculating the spatial geometric intersection of the observation line of the image acquisition unit and the projected light ray of the light source, the candidate three-dimensional coordinates of the surface are located. The occupancy probability of the voxels in the three-dimensional voxel grid is updated according to the candidate three-dimensional coordinates of the surface. Before updating the occupancy probability of voxels in the three-dimensional voxel mesh based on the candidate three-dimensional coordinates of the surface, the method further includes photometric property calculation based on polarization component decoupling: extracting vegetation mask based on the color and polarization information of the active illumination map; and calculating the decoupled diffuse reflection difference map using the following formula with a preset orthogonal polarization layout: ;in, For pixels, For the first i Original observed brightness when the light source is turned on. For ambient background brightness, The angle between the camera polarizer and the polarization direction of the light source is denoted as . The polarization component decoupling coefficient is used; based on the diffuse reflection difference map, the illumination state of a pixel is determined as either high-confidence illumination or pure shadow, and the illumination confidence level is calculated. The light-receiving confidence level Used to update the occupancy probability of voxels in the probability occupancy field update step; The step of updating the occupancy probability of voxels in the three-dimensional voxel mesh based on the candidate three-dimensional coordinates of the surface specifically includes: for the pixels with high confidence in receiving light, calculating the spatial geometric intersection of the camera's line of sight and the light ray projected by the light source. The basic visibility confidence factor acting on voxels in the three-dimensional voxel mesh is calculated using the following formula: ;in, voxels Center to the spatial geometric intersection point The geometric distance, σ is the physical adjustment parameter for compensating for the penumbra at the leaf edge; the basic visibility confidence factor is used to measure the voxel. Update the occupancy probability and output the updated occupancy probability. Using the aforementioned basic visibility confidence factor For voxels Before performing the step of updating the occupancy probability, it also includes a void topology protection correction based on a dynamic confidence gate: for voids located at the intersection of the light source and the spatial geometry. Obtain the light-receiving confidence of voxels along the path. Set a threshold for strong factual evidence. ;like The voxel is determined to be in a state of strong light path penetration, and the final confidence level is directly adjusted accordingly. Equal to the aforementioned basic visibility confidence factor To force the preservation of physical voids; if Then, the sum of the occupancy probabilities of all voxels within the spatial neighborhood of that voxel is calculated. The final confidence level is calculated using the following formula. : ;in, This is a numerical truncation function used to truncate numerical values. Limited to between 0 and 1; Weighting coefficients used to adjust the intensity of the game; To prevent extremely small values where the denominator is zero; through the final confidence level. Update the voxels The probability of occupancy; After traversing all the active lighting maps, a probability occupancy field updated with multi-source lighting information is obtained; based on the probability occupancy field, the voxel set belonging to the seedling entity is determined, and the three-dimensional geometric model of the target seedling is generated using the voxel set.
2. The method according to claim 1, characterized in that, Generate the corrected final confidence level Used to update the voxel The occupancy probability is iteratively updated using a nonlinear decay criterion: ； in, and These represent the voxel occupancy probabilities before and after the update; the iterative update achieves accurate preservation of real physical voids and adaptive softening protection of low-confidence blade edge regions.
3. The method according to claim 1 or 2, characterized in that, Before the step of determining the voxel set belonging to the seedling entity based on the probability occupancy field, a shadow consistency check is also included: For voxels whose probability of occupation exceeds a preset threshold after the initial update Through virtual projection function Simulate its light source in each of the aforementioned calibrated spatial coordinates The theoretical state of light received; The theoretical lighting state is compared with the acquired actual lighting state to calculate the voxel shadow consistency score. ; If the shadow consistency score (Score) The shading consistency score indicates that the voxel has consistent optical path transmission characteristics under multi-source observation. For the voxels The occupancy probability is subject to a mandatory second-weighted correction to remove artifacts left by the neighborhood protection mechanism and output the final probability occupancy field.
4. The method according to claim 3, characterized in that, The steps of determining the voxel set belonging to the seedling entity based on the final probability occupancy field and generating a three-dimensional geometric model of the target seedling using the voxel set specifically include: Set global judgment threshold If the occupancy probability of a voxel after double weighting is higher than If so, the voxel is determined to belong to the leaf body; The moving cube algorithm is used to extract isosurfaces from the final probability occupancy field to generate an initial triangular mesh describing the outer contour of the seedling; The initial triangular mesh is smoothed using Laplacian and morphologically filtered to eliminate the quantization step effect and remove isolated floating fragments, ultimately outputting the optimized three-dimensional geometric model.
5. The method according to claim 1, characterized in that, The method further includes: The image acquisition unit is a global shutter industrial camera with a polarizer, fixed vertically downwards; The multiple light sources on the ring array are high-brightness LED point light sources with polarizers, and their polarization directions are orthogonal to the polarizer direction of the camera. The pre-calibrated spatial coordinates of the light source are obtained by precise calibration of the three-dimensional coordinates of all light sources in the world coordinate system. These three-dimensional coordinates are the geometric reference for constructing the active light and shadow epipolar geometric constraints.
6. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the active lighting and probabilistic occupancy seedling three-dimensional reconstruction method as described in any one of claims 1-5.
7. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of the seedling three-dimensional reconstruction method based on active lighting and probabilistic occupancy as described in any one of claims 1-5.