Focused stack weight domain deep extraction method, device and equipment and storage medium

Abstract

This invention discloses a method, apparatus, device, and storage medium for depth extraction of the focus stack weight domain. The method includes: acquiring a multi-focal plane image sequence; calculating focus sharpness pixel-by-pixel and normalizing it along the focal plane to generate an initial weight map; using the original image as a structural reference to perform edge-preserving filtering on the initial weight map to obtain an optimized weight map; constructing a weight-focal plane curve based on the optimized weight map, and sequentially performing three levels of logical judgment: the first level removes textureless regions based on the maximum weight value; the second level removes high-reflectivity false peaks based on waveform flatness; and the third level filters candidate focal plane positions based on peak focus sharpness; and performing sub-pixel fitting on the selected candidate focal plane positions to convert them into physical depth and generate a 3D point cloud. This invention effectively solves problems such as false points on textureless backgrounds, false peaks on highly reflective surfaces, edge breaks in weak textures, and limited depth accuracy through edge-preserving filtering to optimize the weight map, three-level progressive judgment, and sub-pixel fitting.

Description

Technical Field

[0001] This invention belongs to the field of machine vision 3D reconstruction technology, specifically relating to a method, apparatus, device, and storage medium for focusing stack weight domain depth extraction. Background Technology

[0002] Shape-From-Focus (SFF) 3D reconstruction is a passive optical, non-contact 3D shape restoration technique. Its core principle is that the closer a point on an object's surface is to the focal plane, the sharper the image; the farther away, the blurrier the image. Based on this, SFF technology controls an optical system (such as a precision motorized stage or zoom lens) to acquire a series of 2D images (called a focus stack) with different focal plane positions within the same field of view. Then, image processing algorithms are used to analyze the changes in sharpness pixel by pixel, thereby retrieving the object's 3D height information.

[0003] In industrial applications such as semiconductor packaging, PCB circuit board inspection, and precision hardware measurement, SFF is widely regarded as one of the best candidate technologies for industrial micro-surface inspection due to its unique advantages for highly reflective surfaces, small features, textureless surfaces, and soft workpieces.

[0004] However, existing SFF technology still has the following major drawbacks in industrial applications: First, although there is no height change on the smooth PCB board surface or chip surface, random peaks will be generated due to imaging noise, resulting in a large number of false height points (noise spikes) on the baseboard.

[0005] Second, in highly reflective areas such as gold wires and solder joints, light scattering causes the entire focusing curve to be "high and flat" without a true peak, but existing technology will still forcibly identify an incorrect height.

[0006] Third, fine structures (such as thin wires and characters) have weak focusing response and are easily treated as noise filters, resulting in point cloud breaks or holes.

[0007] Fourth, it relies solely on the maximum response of integer frames and lacks subpixel-level curve analysis capabilities, resulting in limited Z-axis resolution. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a method, apparatus, device, and storage medium for focusing on the depth extraction of stack weight domains.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows: In a first aspect, the present invention discloses a method for focusing on the depth extraction of stack weight domains, comprising: Step S1: Obtain a multi-focal plane image sequence of the object under test. The image sequence includes images at multiple different focal plane positions. Step S2: For each frame in the multi-focal plane image sequence, calculate the focus sharpness pixel by pixel to obtain the focus sharpness value of each pixel at each focal plane position; Step S3: For each pixel, normalize its focus sharpness value along the focal plane direction to obtain the initial membership weight of each pixel at each focal plane position, and generate the initial weight map corresponding to each focal plane. Step S4: Using the original images in the multi-focal plane image sequence as structural references, perform edge-preserving filtering on the initial weight map to obtain the optimized weight map corresponding to each focal plane; Step S5: Based on the optimized weight map corresponding to each focal plane, construct the optimized weight-focal plane curve for each pixel, and perform multi-level logical judgments on the curve sequentially: First-level judgment: Determine whether the maximum weight value of the curve is lower than the preset first energy threshold. If so, determine that the pixel belongs to the textureless region and directly output the reference depth. Second-level judgment: If the maximum weight value is not lower than the first energy threshold, then it is further judged whether the waveform flatness of the curve exceeds the preset second flatness threshold. If so, the pixel is judged to belong to the textureless area and the reference depth is directly output. Third-level judgment: If the waveform flatness does not exceed the second flatness threshold, then identify all local peaks on the curve, calculate the focus sharpness value of each peak, and take the peak with the focus sharpness value higher than the preset third saliency threshold as the candidate focal plane position. Step S6: For the candidate focal plane positions selected by the third-level judgment, perform sub-pixel position fitting to obtain the focal plane position with sub-pixel precision, and convert the focal plane position with sub-pixel precision into physical depth to generate a three-dimensional point cloud.

[0010] Based on the above technical solution, the following improvements can be made: As a preferred approach, edge-preserving filtering is a guided filtering method. The original image at each focal plane position is used as the guiding image, and the initial weight map at the same focal plane position is subjected to guided filtering to obtain an optimized weight map.

[0011] As a preferred approach, guided filtering includes: Step A1: Calculate the window mean μ and window variance σ of the guide map within each preset local window using mean filtering. 2 Window mean of the image to be filtered And the window covariance of the guide graph and the graph to be filtered. ; Where: I is the guiding image, and P is the image to be filtered; Step A2: Based on the window mean, window variance, window covariance, and preset regularization parameters... Calculate the linear relationship coefficients a and b for each pixel within its corresponding local window. The calculation formula is as follows: ; ; Step A3: Apply mean filtering to the linear relationship coefficients a and b corresponding to each window obtained in Step A2, and average the coefficients within all windows containing that pixel at each pixel location to obtain the globally smoothed coefficients. and ; Step A4: Based on the smoothed coefficients and ,according to The optimized weight map is calculated; Where: I is the guiding graph, and Q is the optimized weight graph of the same size as the graph to be filtered.

[0012] As a preferred option, edge-preserving filtering is bilateral filtering.

[0013] As a preferred approach, waveform flatness is measured by calculating the variance or covariance of the weighted focal plane curve in the peak neighborhood.

[0014] As a preferred embodiment, in step S6, the sub-pixel position fitting includes: selecting the candidate focal plane position and the weight values ​​of several adjacent focal plane positions, performing curve fitting, and taking the extreme point of the fitted curve as the focal plane position with sub-pixel accuracy.

[0015] As a preferred option, curve fitting can be any one of quadratic parabolic fitting, Gaussian fitting, or polynomial fitting.

[0016] Secondly, the present invention discloses a device for focusing on stack weight domain depth extraction, comprising: The image acquisition module is used to acquire a multi-focal plane image sequence of the object under test. The image sequence includes images at multiple different focal plane positions. The focus sharpness calculation module is used to calculate the focus sharpness pixel by pixel for each frame in a multi-focal plane image sequence, and obtain the focus sharpness value of each pixel at each focal plane position; The initial weight map generation module is used to normalize the focus sharpness value of each pixel along the focal plane direction to obtain the initial membership weight of each pixel at each focal plane position, and generate the initial weight map corresponding to each focal plane. The optimized weight map generation module uses the original images in the multi-focal plane image sequence as structural references to perform edge-preserving filtering on the initial weight map, thereby obtaining the optimized weight map corresponding to each focal plane. The multi-level logic determination module is used to construct the optimized weight-focal plane curve for each pixel based on the optimized weight map corresponding to each focal plane, and then perform multi-level logic determination on the curve sequentially: First-level judgment: Determine whether the maximum weight value of the curve is lower than the preset first energy threshold. If so, determine that the pixel belongs to the textureless region and directly output the reference depth. Second-level judgment: If the maximum weight value is not lower than the first energy threshold, then it is further judged whether the waveform flatness of the curve exceeds the preset second flatness threshold. If so, the pixel is judged to belong to the textureless area and the reference depth is directly output. Third-level judgment: If the waveform flatness does not exceed the second flatness threshold, then identify all local peaks on the curve, calculate the focus sharpness value of each peak, and take the peak with the focus sharpness value higher than the preset third saliency threshold as the candidate focal plane position. The depth generation module is used to perform sub-pixel position fitting on the candidate focal plane positions selected by the third-level judgment to obtain the focal plane positions with sub-pixel precision, and then convert the focal plane positions with sub-pixel precision into physical depth to generate a 3D point cloud.

[0017] Thirdly, the present invention discloses a computing device, comprising: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-described focused stack weight domain depth extraction methods.

[0018] Fourthly, the present invention discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by memory and executed by any of the above-described focused stack weight domain depth extraction methods.

[0019] This invention discloses a method, apparatus, device, and storage medium for focusing on stack weight domain depth extraction, which has the following beneficial effects: First, this invention obtains a weight map by normalizing the focus sharpness value, and completes pre-optimization in the weight domain by combining edge-preserving filtering. With the addition of two-level judgment of energy threshold and waveform flatness, it can accurately identify textureless areas such as PCB board surface and chip surface, effectively suppress random peaks caused by imaging noise, and completely solve the problem of false height points on the base plate in traditional technology.

[0020] Secondly, for the high and flat focusing curves formed by highly reflective areas such as gold wires and solder joints, this invention uses waveform flatness inspection for identification, instead of forcibly determining the focus position as in existing technologies. This effectively avoids false peaks and false depths, and improves the accuracy of depth extraction for highly reflective workpieces.

[0021] Third, the present invention uses edge-preserving filtering to process the weight map, which can preserve the image edge and detail features while reducing noise. Combined with multi-level logic judgment to filter effective peaks, it avoids the misjudgment of weak textures and fine structures such as thin wires and surface characters as noise, thus solving the defects of three-dimensional point cloud breakage and voids.

[0022] Fourth, after selecting effective candidate focal plane positions, the present invention combines adjacent focal plane data to perform curve fitting, thereby achieving sub-pixel-level focal plane positioning. This breaks through the limitation of traditional schemes that rely solely on the maximum value of integer frames to determine depth, effectively improving the depth resolution in the Z-axis direction and meeting the high-precision requirements of precision industrial inspection.

[0023] Fifth, the present invention adopts a multi-level explicit logic decision-making approach, which has better real-time performance and interpretability compared with deep learning solutions. At the same time, the edge-preserving filtering can be guided filtering or bilateral filtering, and the curve fitting also supports multiple fitting methods, which can be flexibly adapted to different detection scenarios and hardware conditions, making the algorithm more versatile and practical in engineering. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart of a method for extracting depth of the focused stack weight domain provided in an embodiment of the present invention.

[0026] Figure 2 A flowchart for multi-level logic determination provided in an embodiment of the present invention. Detailed Implementation

[0027] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The expression “includes” is an “open-ended” expression, which means that there is a corresponding component or step, and should not be interpreted as excluding additional components or steps.

[0030] like Figures 1 to 2 As shown, this embodiment discloses a method for focusing on stack weight domain depth extraction, including: Step S101: Obtain a multi-focal plane image sequence of the object to be tested, the image sequence including images at multiple different focal plane positions; Step S102: For each frame of the multi-focal plane image sequence, calculate the focus sharpness pixel by pixel to obtain the focus sharpness value of each pixel at each focal plane position; Step S103: For each pixel, normalize its focus sharpness value along the focal plane direction to obtain the initial membership weight of each pixel at each focal plane position, and generate the initial weight map corresponding to each focal plane. Step S104: Using the original images in the multi-focal plane image sequence as structural references, perform edge-preserving filtering on the initial weight map to obtain the optimized weight map corresponding to each focal plane; Step S105: Based on the optimized weight map corresponding to each focal plane, construct the optimized weight-focal plane curve for each pixel, and perform multi-level logical judgments on the curve sequentially: First-level judgment: Determine whether the maximum weight value of the curve is lower than the preset first energy threshold. If so, determine that the pixel belongs to the textureless region and directly output the reference depth. Second-level judgment: If the maximum weight value is not lower than the first energy threshold, then it is further judged whether the waveform flatness of the curve exceeds the preset second flatness threshold. If so, the pixel is judged to belong to the textureless area and the reference depth is directly output. Third-level judgment: If the waveform flatness does not exceed the second flatness threshold, then identify all local peaks on the curve, calculate the focus sharpness value of each peak, and take the peak with the focus sharpness value higher than the preset third saliency threshold as the candidate focal plane position. Step S106: For the candidate focal plane positions selected by the third-level judgment, perform sub-pixel position fitting to obtain the focal plane position with sub-pixel precision, and convert the focal plane position with sub-pixel precision into physical depth to generate a three-dimensional point cloud.

[0031] Each of the above steps will be explained in detail below.

[0032] Step S101: Obtain a multi-focal plane image sequence.

[0033] The lens or stage is driven by a precision stepper motor to move along the Z-axis in fixed step sizes Δz, acquiring N frames of images. Each frame represents a specific focal plane position.

[0034] Step S102: Calculate the focus sharpness value.

[0035] This embodiment uses the Laplacian transform to calculate the second derivative of each frame in a multi-focal plane image sequence, and sums the absolute values ​​of the second derivatives within a local window to obtain the focus sharpness value of each pixel at each focal plane position, thereby measuring the "sharpness" or "high-frequency energy" of the pixel.

[0036] Clear areas have larger response values ​​due to drastic changes in grayscale, while blurry areas have smaller response values.

[0037] Step S103: Normalize to generate the initial weight map.

[0038] For each pixel, its focus sharpness value is normalized along the focal plane direction to generate an initial weight map. Each initial weight map is a two-dimensional matrix of the same size as the original image, with a value range of [0,1]. Here, "1" indicates that the pixel is in the sharpest focus at the current focal plane, and "0" indicates that it is completely out of focus.

[0039] By normalizing, the original focus sharpness value is transformed into a probabilistic weight of the pixel belonging to each focal plane. This makes the weights between different focal planes comparable and lays a numerical foundation for subsequent curve shape analysis (such as peak detection and flatness calculation).

[0040] Step S104: Edge-preserving filtering optimizes the weight graph.

[0041] This embodiment uses guided filtering to achieve edge-preserving filtering. Specifically, let the original image at the current focal plane position be the guiding image I, the initial weight image at the same focal plane position be the image to be filtered P, and the output optimized weight image be denoted as Q.

[0042] The core assumption of guided filtering is that in each local window centered at pixel k... Within (which can be a rectangle), the output image Q and the guiding image I satisfy a linear relationship: (1) in, and For local windows The linear relationship coefficient within; i represents a local window The pixel index within.

[0043] Taking the gradient of both sides of equation (1) yields Therefore, when the guiding graph I has edges, the output Q will maintain the same edge direction, thus achieving structure transfer.

[0044] To solve for the linear relationship coefficients and The goal is to minimize the difference between the output Q and the image P to be filtered. Using the least squares method, the objective function is: (2) in: This is a regularization parameter that determines the smoothness of the filter.

[0045] For the image P to be filtered, in a local window The weight value corresponding to the i-th pixel.

[0046] By solving, a closed-form solution for the coefficients within the window can be obtained: (3) (4) in: , This indicates that the guide image I is in a local window. Mean and variance within; This indicates that the input image P is in a local window. The mean within; For local windows Total number of pixels.

[0047] The above The numerator in the calculation formula is the covariance. .

[0048] Guided filtering specifically includes the following steps: Step A101: Apply mean filtering to a preset local window Calculate separately: 1) Window mean of Guide Figure I and window variance ; 2) Window mean of the image P to be filtered ; 3) Window covariance between the guiding image I and the image to be filtered P .

[0049] Step A102: For each pixel k (corresponding to the window) The linear relationship coefficients of the window are calculated using formulas (3) and (4). and .

[0050] Since each pixel is covered by multiple windows, these coefficients will be processed in subsequent steps.

[0051] Step A103: For each pixel position i, the coefficients after global smoothing and The specific formula is obtained by averaging the window coefficients obtained in step A102: ; ; in: This represents a local window centered at pixel i; It is a local window The total number of pixels within.

[0052] Step A104: Based on the smoothed coefficients, according to The output image Q (i.e., the optimized weight map) is calculated.

[0053] By independently executing the complete process described above for each focal plane, the optimized weight map corresponding to each focal plane can be obtained.

[0054] Guided filtering can suppress noise weights while maintaining the sharpness of object edges, aligning the optimized weight-focal plane curve with the original image at the edges, thus providing high-quality input for subsequent logical decisions.

[0055] It is worth noting that in some other embodiments, the edge-preserving filter can also be a two-sided filter, which will not be elaborated here.

[0056] Step S105: Multi-level logic determination.

[0057] Based on the optimized weight map corresponding to each focal plane, an optimized weight-focal plane curve is constructed for each pixel, and the following three-level logical judgment is performed on the curve in sequence: Level 1 Judgment (Absolute Energy Threshold): For each pixel, calculate the maximum weight value of the curve. Also known as focus energy, this value essentially reflects the focus sharpness of the pixel at its optimal focal plane position.

[0058] Textureless areas (such as smooth PCB surfaces and chip surfaces) have very low sharpness across all focal planes, therefore their Very small; conversely, textured areas will at least achieve a high level of sharpness response at one focal plane.

[0059] This embodiment sets a first energy threshold. (The empirical value range is 0.01 to 0.05, calibrated based on the illumination intensity and material reflectivity; here we take...) ). If the pixel is determined to be at a textureless bottom, a baseline depth is output, and no further determination is made. This stage can quickly eliminate false peaks caused by imaging noise, thus eliminating the "spotting" phenomenon at its source.

[0060] But if Setting the threshold too low, while preserving weak texture areas (such as fine structures under low light), will also leave some noise peaks; setting it too high, weak texture areas will be misjudged as having no texture and thus lost. To resolve this contradiction, this invention does not rely on a single threshold, but introduces a second-level judgment for compensation.

[0061] Second-level judgment (waveform flatness test): For pixels that pass the first-level judgment (i.e. Further analysis was conducted on the flatness of the weight-focal plane curve. Highly reflective surfaces (such as gold wires and solder joints) often exhibit a "high and flat" characteristic in their focusing curves due to light scattering. Although the maximum weight value is relatively large, the curve is generally flat without sharp single peaks. Textureless, low-light areas may also have a slightly higher maximum weight value due to accidental noise. However, its curve is also flat.

[0062] This embodiment uses the variance of the curve within the peak neighborhood (two focal planes to the left and right of the peak point) to quantify flatness, denoted as . The smaller the variance, the flatter the curve. Set a second flatness threshold. .like If the curve is flat, then no matter how high the first-level energy is, the pixel is determined to be a textureless or highly reflective area, and the reference depth is output in the same way.

[0063] By using the second-level judgment, a lower level can be used in the first level. By preserving weak texture signals and using flatness testing to remove false peaks in low-light, textureless areas and high-reflectivity areas, a balance is achieved between "reconstructing low-texture areas and preventing false peaks in textureless areas".

[0064] Level 3 Judgment (Multimodal Significance Screening): When the curve passes both of the first two levels of judgment (i.e.) The curve is not flat, indicating that the pixel does indeed have one or more true focus peaks. However, due to the limited depth of field of the optical system and the complex structure of the object's surface, multiple local peaks may appear on a single curve. For example, a thin guideline (weak texture) and its strongly reflective base below it will produce two peaks during the defocus-focus process: when the guideline is in focus, the base is out of focus, and when the base is in focus, the guideline is out of focus, forming two peaks of different heights on the curve. The traditional WTA algorithm simply selects the highest peak, leading to incorrect depth jumps (misjudging the height of the guideline as the height of the base or vice versa), resulting in severe artifacts.

[0065] The third-level determination in this embodiment addresses this issue by calculating the salience of each local peak. Here, the salience is directly taken from the original focus sharpness value corresponding to the peak location, as it physically represents the true high-frequency energy of the image at that focal plane. A third salience threshold is set. For each local peak on the curve, if its focus sharpness value is higher than... If the value is positive, it is retained as a candidate focal plane position; otherwise, it is discarded. In this way, even if a certain peak is not the highest on the weight-focal plane curve (such as the peak corresponding to the fine guide), as long as its focus sharpness value is high enough (significantly higher than background noise and local troughs), it can be retained, thereby reconstructing the shape of the fine guide and the base at the same time.

[0066] Through the aforementioned three-level progressive judgment, this invention completely abandons the traditional WTA's "maximum value only" approach, achieving a complete analysis and intelligent decision-making of the focus response curve morphology. Actual testing shows that for PCB samples containing smooth base plates, highly reflective gold lines, and fine characters, the reconstructed 3D point cloud generated by this method shows no false points on the base plate, no false peaks in the gold lines, clear and continuous character edges, and a significant reduction in depth jump errors at complex texture overlaps.

[0067] Step S106: Subpixel position fitting and 3D point cloud generation.

[0068] For the candidate focal plane positions selected in the third-level judgment, sub-pixel position fitting is performed. Specifically, the weight values ​​of the candidate peak points and their adjacent focal plane positions are extracted from the optimized weight curve. In this embodiment, a three-point quadratic parabola fitting is preferably used, that is, the candidate peak point and one adjacent point on each side are selected, and the extreme point is taken as the focal plane position with sub-pixel accuracy after fitting the parabola. It should be noted that the specific implementation of curve fitting is not limited to a three-point quadratic parabola. More points (such as five points) can also be used for Gaussian fitting or polynomial fitting. The number of fitting points and the type of fitting function can be flexibly adjusted according to the actual curve shape and accuracy requirements.

[0069] The focal plane position with subpixel precision is converted into physical depth, thereby generating a 3D point cloud.

[0070] In other embodiments, the present invention discloses a focused stack weight domain depth extraction apparatus, comprising: The image acquisition module is used to acquire a multi-focal plane image sequence of the object under test. The image sequence includes images at multiple different focal plane positions. The focus sharpness calculation module is used to calculate the focus sharpness pixel by pixel for each frame in a multi-focal plane image sequence, and obtain the focus sharpness value of each pixel at each focal plane position; The initial weight map generation module is used to normalize the focus sharpness value of each pixel along the focal plane direction to obtain the initial membership weight of each pixel at each focal plane position, and generate the initial weight map corresponding to each focal plane. The optimized weight map generation module uses the original images in the multi-focal plane image sequence as structural references to perform edge-preserving filtering on the initial weight map, thereby obtaining the optimized weight map corresponding to each focal plane. The multi-level logic determination module is used to construct the optimized weight-focal plane curve for each pixel based on the optimized weight map corresponding to each focal plane, and then perform multi-level logic determination on the curve sequentially: First-level judgment: Determine whether the maximum weight value of the curve is lower than the preset first energy threshold. If so, determine that the pixel belongs to the textureless region and directly output the reference depth. Second-level judgment: If the maximum weight value is not lower than the first energy threshold, then it is further judged whether the waveform flatness of the curve exceeds the preset second flatness threshold. If so, the pixel is judged to belong to the textureless area and the reference depth is directly output. Third-level judgment: If the waveform flatness does not exceed the second flatness threshold, then identify all local peaks on the curve, calculate the focus sharpness value of each peak, and take the peak with the focus sharpness value higher than the preset third saliency threshold as the candidate focal plane position. The depth generation module is used to perform sub-pixel position fitting on the candidate focal plane positions selected by the third-level judgment to obtain the focal plane positions with sub-pixel precision, and then convert the focal plane positions with sub-pixel precision into physical depth to generate a 3D point cloud.

[0071] In other embodiments, the present invention discloses a computing device comprising: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include instructions for any of the above-described focused stack weight domain depth extraction methods.

[0072] Fourthly, the present invention discloses a storage medium storing one or more computer-readable programs, the one or more programs including instructions adapted to be loaded by memory and executed by any of the above-described focused stack weight domain depth extraction methods.

[0073] This invention discloses a method, apparatus, device, and storage medium for focusing on stack weight domain depth extraction, which has the following beneficial effects: First, this invention obtains a weight map by normalizing the focus sharpness value, and completes pre-optimization in the weight domain by combining edge-preserving filtering. With the addition of two-level judgment of energy threshold and waveform flatness, it can accurately identify textureless areas such as PCB board surface and chip surface, effectively suppress random peaks caused by imaging noise, and completely solve the problem of false height points on the base plate in traditional technology.

[0074] Secondly, for the high and flat focusing curves formed by highly reflective areas such as gold wires and solder joints, this invention uses waveform flatness inspection for identification, instead of forcibly determining the focus position as in existing technologies. This effectively avoids false peaks and false depths, and improves the accuracy of depth extraction for highly reflective workpieces.

[0075] Third, the present invention uses edge-preserving filtering to process the weight map, which can preserve the image edge and detail features while reducing noise. Combined with multi-level logic judgment to filter effective peaks, it avoids the misjudgment of weak textures and fine structures such as thin wires and surface characters as noise, thus solving the defects of three-dimensional point cloud breakage and voids.

[0076] Fourth, after selecting effective candidate focal plane positions, the present invention combines adjacent focal plane data to perform curve fitting, thereby achieving sub-pixel-level focal plane positioning. This breaks through the limitation of traditional schemes that rely solely on the maximum value of integer frames to determine depth, effectively improving the depth resolution in the Z-axis direction and meeting the high-precision requirements of precision industrial inspection.

[0077] Fifth, the present invention adopts a multi-level explicit logic decision-making approach, which has better real-time performance and interpretability compared with deep learning solutions. At the same time, the edge-preserving filtering can be guided filtering or bilateral filtering, and the curve fitting also supports multiple fitting methods, which can be flexibly adapted to different detection scenarios and hardware conditions, making the algorithm more versatile and practical in engineering.

[0078] Furthermore, it should be noted that the depth extraction device for the focused stack weight domain provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the focused stack weight domain depth extraction device can be divided into different functional modules to complete all or part of the functions described above.

[0079] Furthermore, the embodiments of the focused stack weight domain depth extraction device and the focused stack weight domain depth extraction method provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0080] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A method for focused stack weight domain deep extraction, characterized in that, include: Step S1: Obtain a multi-focal plane image sequence of the object to be tested, wherein the image sequence includes images at multiple different focal plane positions; Step S2: For each frame of the multi-focal plane image sequence, calculate the focus sharpness pixel by pixel to obtain the focus sharpness value of each pixel at each focal plane position; Step S3: For each pixel, normalize its focus sharpness value along the focal plane direction to obtain the initial membership weight of each pixel at each focal plane position, and generate the initial weight map corresponding to each focal plane. Step S4: Using the original images in the multi-focal plane image sequence as structural references, perform edge-preserving filtering on the initial weight map to obtain the optimized weight map corresponding to each focal plane; Step S5: Based on the optimized weight map corresponding to each focal plane, construct the optimized weight-focal plane curve for each pixel, and perform multi-level logical judgments on the curve sequentially: First-level judgment: Determine whether the maximum weight value of the curve is lower than the preset first energy threshold. If so, determine that the pixel belongs to the textureless region and directly output the reference depth. Second-level judgment: If the maximum weight value is not lower than the first energy threshold, then it is further judged whether the waveform flatness of the curve exceeds the preset second flatness threshold. If so, the pixel is judged to belong to the textureless area and the reference depth is directly output. Third-level judgment: If the waveform flatness does not exceed the second flatness threshold, then identify all local peaks on the curve, calculate the focus sharpness value of each peak, and take the peak with the focus sharpness value higher than the preset third saliency threshold as the candidate focal plane position. Step S6: For the candidate focal plane positions selected by the third-level judgment, perform sub-pixel position fitting to obtain the focal plane position with sub-pixel precision, and convert the focal plane position with sub-pixel precision into physical depth to generate a 3D point cloud.

2. The method of claim 1, wherein, The edge-preserving filter is a guided filter, which uses the original image of each focal plane position as the guide image and performs guided filtering on the initial weight image of the same focal plane position to obtain an optimized weight image.

3. The method according to claim 2, characterized in that, The guided filtering includes: Step A1: Calculate the window mean μ and window variance σ of the guide map within each preset local window using mean filtering. 2 Window mean of the image to be filtered And the window covariance of the guide graph and the graph to be filtered. ; Where: I is the guiding image, and P is the image to be filtered; Step A2: Based on the window mean, window variance, window covariance, and preset regularization parameters... Calculate the linear relationship coefficients a and b for each pixel within its corresponding local window. The calculation formula is as follows: ； ； Step A3: Perform mean filtering on the linear relationship coefficients a and b corresponding to each window obtained in Step A2, and average the coefficients within all windows containing that pixel at each pixel location to obtain the globally smoothed coefficients. and ; Step A4: Based on the smoothed coefficients and ,according to The optimized weight map is calculated; Where: I is the guiding graph, and Q is the optimized weight graph of the same size as the graph to be filtered.

4. The method according to claim 1, characterized in that, The edge-preserving filter is a bilateral filter.

5. The method according to claim 1, characterized in that, The waveform flatness is measured by calculating the variance or covariance of the weighted focal plane curve in the peak neighborhood.

6. The method according to claim 1, characterized in that, In step S6, the sub-pixel position fitting includes: selecting the weight values ​​of the candidate focal plane position and several adjacent focal plane positions, performing curve fitting, and taking the extreme point of the fitted curve as the focal plane position with sub-pixel accuracy.

7. The method according to claim 6, characterized in that, The curve fitting can be any one of quadratic parabolic fitting, Gaussian fitting, or polynomial fitting.

8. A device for focusing on stack weight domain depth extraction, characterized in that, include: The image acquisition module is used to acquire a multi-focal plane image sequence of the object under test, wherein the image sequence includes images at multiple different focal plane positions; The focus sharpness calculation module is used to calculate the focus sharpness pixel by pixel for each frame of the multi-focal plane image sequence, and obtain the focus sharpness value of each pixel at each focal plane position; The initial weight map generation module is used to normalize the focus sharpness value of each pixel along the focal plane direction to obtain the initial membership weight of each pixel at each focal plane position, and generate the initial weight map corresponding to each focal plane. The optimized weight map generation module is used to use the original images in the multi-focal plane image sequence as structural references to perform edge-preserving filtering on the initial weight map to obtain the optimized weight map corresponding to each focal plane. The multi-level logic determination module is used to construct the optimized weight-focal plane curve for each pixel based on the optimized weight map corresponding to each focal plane, and then perform multi-level logic determination on the curve sequentially: First-level judgment: Determine whether the maximum weight value of the curve is lower than the preset first energy threshold. If so, determine that the pixel belongs to the textureless region and directly output the reference depth. Second-level judgment: If the maximum weight value is not lower than the first energy threshold, then it is further judged whether the waveform flatness of the curve exceeds the preset second flatness threshold. If so, the pixel is judged to belong to the textureless area and the reference depth is directly output. Third-level judgment: If the waveform flatness does not exceed the second flatness threshold, then identify all local peaks on the curve, calculate the focus sharpness value of each peak, and take the peak with the focus sharpness value higher than the preset third saliency threshold as the candidate focal plane position. The depth generation module is used to perform sub-pixel position fitting on the candidate focal plane positions selected by the third-level judgment to obtain the focal plane positions with sub-pixel precision, and then convert the focal plane positions with sub-pixel precision into physical depth to generate a 3D point cloud.

9. A computing device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for the focused stack weight domain depth extraction method as described in any of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores one or more computer-readable programs, the programs including instructions adapted to be loaded by memory and executed as described in any of claims 1-7.

Images