Object depth map generation method, computer equipment, storage medium and program product

Abstract

The invention discloses an object depth map generation method, computer equipment, a storage medium and a program product. The method comprises the following steps: moving an objective table along an optical axis to collect a multi-frame image stack; performing complex value wavelet transform on the stack frame by frame, and determining a candidate layer number on a multi-scale sub-band according to an amplitude maximum principle; selecting a plurality of scales with the highest frequency, aligning the corresponding layer number images to the original resolution through nearest neighbor upsampling, and forming a layer number voting set of each pixel; generating an initial seed according to the same-layer ticket number threshold value and the two adjacent-layer ticket combination threshold values; and under the constraint of a layer difference threshold value, a step-by-step growth strategy based on vote confidence from high to low is adopted. According to the method, staggered layer penetration and fragmentation artifacts are effectively inhibited through'high confidence priority + layer difference continuity constraint ', the coverage and edge retention capability of a low-texture region is improved, three-dimensional height information of an object can be represented with high precision, and the method is suitable for scenes such as multi-focal fusion and microscopic imaging.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and specifically relates to multifocal microscopy imaging technology. Background Technology

[0002] Due to the physical optical limitations of imaging systems, the depth of field of a single image is limited. Areas outside the focal plane exhibit significant defocus blur, resulting in loss of detail and texture, making it difficult to meet the requirements for clear imaging and dimensional measurement of targets at all heights. By moving the stage along the optical axis to acquire multiple images at different focal lengths and stacking them, and then processing the stack to generate a fully sharp image and corresponding depth map, the depth of field of the imaging system can be extended, while providing quantitative and continuous geometric information for 3D reconstruction, shape assessment, automatic detection, and measurement.

[0003] Existing methods for recovering depth from stacked images mainly include: pixel-wise maximum selection based on sharpness measures, such as gradient maximum selection and local variance maximum selection, which finds the image that maximizes the measure at each pixel and uses its corresponding focal length as the depth; shape self-focusing methods that add spatial regularization to this approach, which smooth and denoise the initial depth through windowing or global optimization; multi-scale methods that construct activity or saliency maps on the image pyramid to assist in depth determination; and in recent years, depth estimation schemes that use neural networks to learn focal length cues have also emerged. The above methods generally face the following problems in practical applications: the sharpness confidence of low texture and low signal-to-noise ratio is generally low; pixel-level point-by-point decision-making is prone to generating a large number of "invalid" or uncertain labels, forming holes, which require post-processing and strong smoothing; depth inference relying solely on local extrema may still produce significant misjudgments, such as layer number jumps, boundary jaggedness, and halos, especially in areas with rapid changes in the focal plane or thin structures, affecting the continuity of the depth map and the reliability of measurement; deep learning methods are highly dependent on the consistency of large-scale labeled data and imaging conditions, have limited cross-device and cross-sample domain out-of-domain generalization ability, high training and deployment costs, and insufficient interpretability, making it difficult to directly meet the industrial application requirements of metrological traceability and parameter controllability. Summary of the Invention

[0004] To address the technical problems mentioned in the background section, this invention proposes a method for generating object depth maps, a computer device, a storage medium, and a program product.

[0005] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows: A method for generating an object depth map includes the following steps: (1) Obtain the image stack I obtained by moving the stage from bottom to top along the optical axis. k The subscript k is the image index, k=1,2,...K, where K is the total number of images; for stack Ik Each image in the dataset undergoes multi-scale complex-valued wavelet decomposition to obtain K complex-valued wavelet images W. k ; (2) With W k The magnitude of the complex number at each point is used as the focal length response. For each pixel position, the layer number corresponding to the point with the largest magnitude in K images is found and saved to the maximum value layer number map. (3) Select the N highest frequency scale layers from the wavelet sub-bands in the maximum value layer number map, extract the corresponding candidate layer number sub-maps for the HL, LH, and HH directions of each scale, and upsample them to the level of I. k With the same resolution, each pixel is mapped one-to-one with the full-resolution pixel, resulting in 3N candidate layer number images, which together form the layer number voting set V for each pixel's location. 3N ; (4) Examine the voting set V for each pixel layer number. 3N For each of the 3N layer numbers, the layer number and corresponding votes of the same layer appearing in each pixel, the layer numbers of two consecutive layers appearing in the same layer and the corresponding combined votes are counted. The candidate layer k1 with the highest number of votes is recorded in BestSeeds, and the corresponding votes v1 are recorded in BestVotes. The larger layer k2 in the pair of two consecutive layers with the highest combined votes is recorded in SecondSeeds, and the combined votes v2 are recorded in SecondVotes. (5) Generate initial seeds based on BestSeeds, BestVotes, SecondSeeds, and SecondVotes obtained in step (4), and initialize the depth map; (6) Construct a set of vote confidence thresholds, and based on the set of vote confidence thresholds, adopt a stepwise growth strategy of vote confidence from high to low for the depth map, and generate the final depth map through several iterations.

[0006] Furthermore, in step (3), the nearest neighbor interpolation method is used to upsample the candidate layer subgraph to the level of I. k Same resolution.

[0007] Further, in step (5), for pixels with a vote count v1 ≥ T1 in BestVotes, they are marked as initial seeds and the corresponding layer number k1 in BestSeeds is written into the depth map; for pixels with a combined vote count v2 ≥ T2 in SecondVotes, they are also marked as initial seeds and the corresponding layer number k2 in SecondSeeds is written into the depth map; where T1 and T2 are preset thresholds.

[0008] Further, in step (5), a vote confidence threshold T is set for each iteration. The threshold for the first round is the highest threshold. The set of pixels recorded in the current depth map is denoted as {S}, and the initial {S} is the initial seed set obtained in step (5). All unrecorded pixels p(i,j) within the neighborhood of the points in {S} are searched in the depth map. If the position (i,j) of p(i,j) is in the set V, 3N There exists a certain layer k i highest number of votes v i ≥T, and there exists a layer number k for any point in {S} adjacent to point p(i,j). s If the absolute value of the difference between the point and k0 is less than the preset value, then the point is recorded and the layer number k0 is assigned to the corresponding position in the depth map. At the same time, the point is added to {S}. If at least one pixel is recorded under the current threshold T, it is considered that "growth" has occurred. The current threshold T is kept unchanged, and the next round of iteration continues. Otherwise, it is considered that "growth" has not occurred. The threshold T is reduced to the second highest threshold, and the next round of iteration continues. This cycle continues until the threshold T is reduced to the lowest threshold and no new pixels are accepted, at which point the iteration ends.

[0009] Furthermore, if set V 3N At position (i,j), there exist multiple layers of k0,...,k n The highest number of votes v0,...,v n If all values ​​are equal and greater than or equal to the current threshold T, then multiple layers k0,...,k are selected preferentially. n The value of layer number k s The layer with the smallest absolute value of the difference is recorded; if there are multiple layers k0,...,k n The value of layer number k s If the absolute values ​​of the differences are still the same, then the one with the larger layer number is recorded.

[0010] Furthermore, in step (5), a set of vote confidence thresholds T, from high to low, is constructed. H ,T H -1,...,T L T H T is the highest threshold. H =[0.8×3N],T L As the minimum threshold, T L =[0.2×3N], [*] indicates rounding up.

[0011] Secondly, the present invention also provides a computer device, which includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the method for generating the object depth map.

[0012] Furthermore, the present invention also provides a computer storage medium storing a computer program that, when executed on a processor, implements the method for generating the object depth map.

[0013] Finally, the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements a method for generating a depth map of the object.

[0014] The beneficial effects of adopting the above technical solution are as follows: This invention utilizes a multi-scale, multi-directional layer number voting set, combined with a threshold for the number of votes within the same layer and a threshold for merging votes between adjacent layers, to improve the reliability of judgment in low-texture and noisy regions. It also provides better perception of the focal plane transition zone, reducing holes and layer infiltration. During the growth process, a continuity constraint of the layer difference threshold is introduced, accepting only pixels with layer numbers compatible with the already grown region, effectively reducing layer number jumps, jagged edges, and halos, while maintaining overall smoothness and structural edge sharpness. A scheduling strategy of "returning to a high threshold when growth occurs, and gradually decreasing to a lower threshold when no growth occurs" is adopted, prioritizing the expansion of high-confidence pixels while avoiding excessive rejection of medium-confidence regions, thus improving convergence speed and coverage. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0017] This embodiment relates to a method for generating an object depth map, such as... Figure 1 As shown.

[0018] The first step is to acquire the image stack I obtained by moving the stage from bottom to top along the optical axis. k There are a total of K images. Then, each image in the stack is subjected to Complex Wavelet Transform (CWT) at multiple scales, resulting in K complex wavelet images W. k .

[0019] The second step is to use W k The magnitude of the complex number at each point is used as the focal length response. For each pixel location, the layer number corresponding to the point with the largest magnitude in K images is found and saved to the maximum value layer number map MaxLayer.

[0020] The third step is to select the N highest-frequency scale layers (typically N=4) from the wavelet subbands of MaxLayer. For each scale s, extract the corresponding candidate layer number sub-images (the sub-band size is 1 / 2 of the original image) from the three directional subbands (HL, LH, HH). s), and upsampled to I using nearest neighbor interpolation. k The same original resolution is used to ensure a one-to-one correspondence between pixels and the full-resolution pixels. This results in 3N candidate layer number images with three dimensions × N, forming a layer number voting set V for each pixel's location. 3N .

[0021] The fourth step is to examine the voting set V for each pixel layer number. 3N The 3N layer numbers are used to count the same layer number and the corresponding number of votes in each pixel, the layer numbers and the combined number of votes of two consecutive layers, and record the candidate layer k1 with the highest number of votes to BestSeeds and its votes v1 to BestVotes; the larger layer k2 of the pair of adjacent layers with the highest combined number of votes is recorded in SecondSeeds, and the combined number of votes v2 is recorded in SecondVotes.

[0022] The fifth step involves generating initial seeds and initializing the depth map based on the highest vote count map and the combined vote count map of adjacent layers. Specifically, for pixels in BestVotes where the highest vote count v1 ≥ T1 (typically 3N*85%), they are marked as seeds, and the layer number k1 of BestSeeds is written into the depth map; similarly, for pixels in SecondVotes where the combined vote count v2 ≥ T2 (typically 3N*90%), they are also marked as seeds, and the layer number k2 of SecondSeeds is written into the depth map. This yields the seed set and its initial labels.

[0023] Step 6: Construct a set of confidence thresholds T for votes from high to low. H ,T H -1,...,T L T H T is the highest threshold. H =[0.8×3N],T L As the minimum threshold, T L =[0.2×3N], [*] indicates rounding up.

[0024] Step 7: The DepthMap is grown progressively from highest to lowest "vote confidence level". Specifically, a vote confidence level threshold T is set for each iteration, with the first iteration using the highest threshold T. H Let {S} be the set of pixels already recorded in the current depth map, and let {S} be the initial seed set obtained in step 5. Search within the depth map for all unrecorded pixels p(i,j) within the neighborhood of points in {S}. If the position (i,j) of p(i,j) is within the set V... 3N There exists a certain layer k i highest number of votes v i≥T, and there exists a layer number k for any point in {S} adjacent to point p(i,j). s If the absolute value of the difference between k0 and the threshold is less than a preset value (typically 2), then the point is recorded and the layer number k0 is assigned to the corresponding position in the depth map. Simultaneously, the point is added to {S}. If at least one pixel is recorded at the current threshold T, it is considered that "growth" has occurred, the current threshold T remains unchanged, and the next iteration continues. Otherwise, it is considered that "growth" has not occurred, and the threshold T is reduced to T. H -1, and continue to the next iteration; repeat this process until the threshold T decreases to the minimum threshold T. L And no new pixels are accepted, the iteration ends.

[0025] To make the above stepwise growth strategy more complete, if set V 3N At position (i,j), there exist multiple layers of k0,...,k n The highest number of votes v0,...,v n If all values ​​are equal and greater than or equal to the current threshold T, then multiple layers k0,...,k are selected preferentially. n The value of layer number k s The layer with the smallest absolute value of the difference is recorded; if there are multiple layers k0,...,k n The value of layer number k s If the absolute values ​​of the differences are still the same, then the one with the larger layer number is recorded.

[0026] Furthermore, the present invention also relates to a computer device comprising a processor and a memory, the memory storing a computer program, the processor being configured to execute the computer program to implement the method for generating the object depth map.

[0027] Furthermore, the present invention also relates to a computer storage medium storing a computer program that, when executed on a processor, implements a method for generating the depth map of the object.

[0028] Finally, the present invention also relates to a computer program product, including a computer program or instructions that, when executed by a processor, implement a method for generating a depth map of the object.

[0029] It will be understood by those skilled in the art that this invention may relate to apparatus for performing one or more of the operations described in this application. The apparatus may be specifically designed and manufactured for the desired purpose, or may include known devices in general-purpose computers with programs stored therein that can be selectively activated or reconfigured. Such computer programs may be stored in a device (e.g., computer)-readable medium or in any type of medium suitable for storing electronic instructions and coupled to a bus, including but not limited to any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), random access memory (RAM), read-only memory (ROM), electrically programmable ROM, electrically erasable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic cards, or optical cards. A readable medium includes any mechanism for storing or transmitting information in a form readable by a device (e.g., computer). For example, readable media include random access memory (RAM), read-only memory (ROM), disk storage media, optical storage media, flash memory devices, signals propagated in electrical, optical, acoustic, or other forms (e.g., carrier waves, infrared signals, digital signals), etc.

[0030] Those skilled in the art will understand that each box in these structure diagrams and / or block diagrams and / or flow diagrams, as well as combinations of boxes in these structure diagrams and / or block diagrams and / or flow diagrams, can be implemented using computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing method to generate a machine, thereby creating, through execution by the processor of the computer or other programmable data processing method, methods specified in the boxes of the structure diagrams and / or block diagrams and / or flow diagrams.

[0031] It will be understood by those skilled in the art that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that the term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0032] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0033] The embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for generating an object depth map, characterized in that, Includes the following steps: (1) Obtain the image stack I obtained by moving the stage from bottom to top along the optical axis. k The subscript k is the image index, k=1,2,...K, where K is the total number of images; for stack I k Each image in the dataset undergoes multi-scale complex-valued wavelet decomposition to obtain K complex-valued wavelet images W. k ; (2) With W k The magnitude of the complex number at each point is used as the focal length response. For each pixel position, the layer number corresponding to the point with the largest magnitude in K images is found and saved to the maximum value layer number map. (3) Select the N highest frequency scale layers from the wavelet sub-bands in the maximum value layer number map, extract the corresponding candidate layer number sub-maps for the HL, LH, and HH directions of each scale, and upsample them to the level of I. k With the same resolution, each pixel is mapped one-to-one with the full-resolution pixel, resulting in 3N candidate layer number images, which together form the layer number voting set V for each pixel's location. 3N ; (4) Examine the voting set V for each pixel layer number. 3N The 3N layer numbers in the data are used to count the same layer number and the corresponding number of votes in each pixel, as well as the layer numbers of two consecutive layers and the corresponding combined number of votes. The candidate layer k1 with the highest number of votes is recorded in BestSeeds, and the corresponding number of votes v1 is recorded in BestVotes. Record the larger layer k2 in the pair of consecutive layers with the highest combined vote count into SecondSeeds, and record the combined vote count v2 into SecondVotes; (5) Generate initial seeds based on BestSeeds, BestVotes, SecondSeeds, and SecondVotes obtained in step (4), and initialize the depth map; (6) Construct a set of vote confidence thresholds, and based on the set of vote confidence thresholds, adopt a stepwise growth strategy of vote confidence from high to low for the depth map, and generate the final depth map through several iterations.

2. The method for generating an object depth map according to claim 1, characterized in that, In step (3), the nearest neighbor interpolation method is used to upsample the candidate layer subgraph to the level of I. k Same resolution.

3. The method for generating an object depth map according to claim 1, characterized in that, In step (5), for pixels with a vote count v1 ≥ T1 in BestVotes, they are marked as initial seeds and the corresponding layer number k1 in BestSeeds is written into the depth map; for pixels with a combined vote count v2 ≥ T2 in SecondVotes, they are also marked as initial seeds and the corresponding layer number k2 in SecondSeeds is written into the depth map. T1 and T2 are preset thresholds.

4. The method for generating an object depth map according to claim 1, characterized in that, In step (5), a vote confidence threshold T is set for each iteration. The threshold for the first round is the highest threshold. The set of pixels recorded in the current depth map is denoted as {S}. The initial {S} is the initial seed set obtained in step (5). All unrecorded pixels p(i,j) within the neighborhood of the points in {S} are searched in the depth map. If the position (i,j) of p(i,j) is in the set V, the confidence threshold is determined. 3N There exists a certain layer k i highest number of votes v i ≥T, and there exists a layer number k for any point in {S} adjacent to point p(i,j). s If the absolute value of the difference between the point and k0 is less than the preset value, then the point is recorded and the layer number k0 is assigned to the corresponding position in the depth map. At the same time, the point is added to {S}. If at least one pixel is recorded under the current threshold T, it is considered that "growth" has occurred. The current threshold T is kept unchanged, and the next round of iteration continues. Otherwise, it is considered that no "growth" has occurred. The threshold T is reduced to the second highest threshold, and the next round of iteration continues. This cycle continues until the threshold T is reduced to the lowest threshold and no new pixels are accepted, at which point the iteration ends.

5. The method for generating an object depth map according to claim 4, characterized in that, If set V 3N At position (i,j), there exist multiple layers of k0,...,k n The highest number of votes v0,...,v n If all values ​​are equal and greater than or equal to the current threshold T, then multiple layers k0,...,k are selected preferentially. n The value of layer number k s The layer with the smallest absolute value of the difference is recorded; if there are multiple layers k0,...,k n The value of layer number k s If the absolute values ​​of the differences are still the same, then the one with the larger layer number is recorded.

6. The method for generating an object depth map according to claim 1, characterized in that, In step (5), a set of vote confidence thresholds T from high to low is constructed. H ,T H -1,...,T L T H T is the highest threshold. H =[0.8×3N],T L As the minimum threshold, T L =[0.2×3N], [*] indicates rounding up.

7. A computer device, the computer device comprising a processor and a memory, the memory storing a computer program, characterized in that: The processor is used to execute the computer program to implement the method for generating an object depth map according to any one of claims 1-6.

8. A computer storage medium storing a computer program, characterized in that: When the computer program is executed on a processor, it implements the method for generating an object depth map as described in any one of claims 1-6.

9. A computer program product, comprising a computer program or instructions, characterized in that: When the computer program or instructions are executed by a processor, they implement the method for generating an object depth map as described in any one of claims 1-6.

Images