Depth map reference plane calibration method, device, equipment and storage medium

Abstract

The application discloses a depth map reference surface calibration method, device, equipment and storage medium. The method first acquires an initial depth map of a target object; respectively, a first depth distribution feature of the initial depth map in a full map area and a second depth distribution feature of the initial depth map in a preset frame area are counted; joint analysis is performed based on the first depth distribution feature and the second depth distribution feature to determine a candidate base layer interval; a sampling mask is generated in the depth map according to the candidate base layer interval and non-base area information of the target object; depth data sampling is performed in a region covered by the sampling mask to obtain a base sample point set; geometric fitting is performed on the base sample point set to obtain a reference surface geometric model; and height calibration processing is performed on the depth map by using the reference surface geometric model to obtain a target calibration depth map. The application can improve the accuracy of depth map zero height reference surface estimation, and in turn reduce systematic measurement error.

Description

Technical Field

[0001] This application relates to the fields of computer vision and industrial intelligent inspection technology, and in particular to a depth map reference surface calibration method, apparatus, device and storage medium. Background Technology

[0002] In industrial 3D vision systems, techniques such as structured light or laser triangulation are typically used to acquire height information of the target surface. Based on the reconstructed depth map, tasks such as solder paste volume measurement, pin coplanarity detection, and device height evaluation are performed. In this process, reference plane calibration of the depth map is a geometric preprocessing step, and the accuracy of its zero-height reference directly affects subsequent height determination and volume calculation. Traditional reference plane calibration methods usually rely on manually pre-specified regions of interest (ROIs) or predefined substrate color features, and use the extracted data for plane fitting.

[0003] In related technologies, existing reference surface calibration schemes rely heavily on manually defined prior information. When faced with conditions such as partial product occlusion, board warping, and model changes, it is difficult to obtain an accurate zero-height reference surface stably, resulting in large systematic measurement errors in subsequent measurements. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a depth map reference surface calibration method, apparatus, device, and storage medium, which can improve the accuracy of depth map zero-height reference surface estimation, thereby reducing systematic measurement errors.

[0005] In a first aspect, embodiments of this application provide a depth map reference surface calibration method, including: Obtain the initial depth map of the target object; The first depth distribution features of the initial depth map within the entire map area and the second depth distribution features within the preset border area are statistically analyzed respectively. Based on the joint analysis of the first depth distribution features and the second depth distribution features, candidate basal layer intervals are determined; Based on the candidate basal layer interval and the non-basal region information of the target object, a sampling mask is generated in the depth map; Depth data is sampled within the area covered by the sampling mask to obtain a base sample point set; Geometric fitting is performed on the base sample point set to obtain a reference surface geometric model; The depth map is height-calibrated using the reference surface geometric model to obtain a target calibration depth map.

[0006] According to some embodiments of this application, the step of separately statistically analyzing the first depth distribution features of the initial depth map within the entire map region and the second depth distribution features within the preset border region includes: Within a preset depth range, the full-map histogram of the initial depth map within the full-map area is calculated and used as the first depth distribution feature; Calculate the border histogram of the initial depth map within the preset border area, and use it as the second depth distribution feature; Based on the joint analysis of the first depth distribution features and the second depth distribution features, candidate basal layer intervals are determined, including: Cluster analysis is performed based on the full image histogram to determine the full image depth cluster set, and cluster analysis is performed based on the border histogram to determine the border depth cluster set; Extract the target full-map depth cluster containing the largest number of data points from the full-map depth cluster set; In the set of border depth clusters, find the local border depth cluster whose depth range is lower than the target full image depth cluster and whose depth position is the highest among the remaining border depth clusters; The depth interval corresponding to the local bounding box depth cluster is determined as the candidate basal layer interval.

[0007] According to some embodiments of this application, after determining the candidate basal layer interval based on the joint analysis of the first depth distribution features and the second depth distribution features, the method further includes: Calculate the support ratio of the pixels corresponding to the candidate base layer interval within the preset border area; If the support ratio is lower than a preset ratio threshold, the preset noise filtering parameter is increased, and the first depth distribution feature, the second depth distribution feature, and the candidate basal layer interval are recalculated; wherein, the increased noise filtering parameter is used to remove depth data with lower frequency during histogram statistics and cluster analysis.

[0008] According to some embodiments of this application, generating a sampling mask in the initial depth map based on the candidate basal layer interval and the non-basal region information of the target object includes: Extract pixels whose depth values ​​are located within the candidate basal layer interval from the initial depth map to generate an initial mask; Obtain the device metadata of the target object, determine the device location region of the target object based on the device metadata, and use the device location region as the non-substrate region; The sampling mask is obtained by removing pixels belonging to the non-base region from the initial mask.

[0009] According to some embodiments of this application, the step of performing depth data sampling within the area covered by the sampling mask to obtain a base sample point set includes: Within the sampling mask, grid sampling is performed at preset physical intervals to obtain a set of grid sampling points; The effective sampling rate is obtained by counting the number of valid sampling points that fall within the effective area of ​​the sampling mask and calculating the ratio of the number of valid sampling points to the total number of theoretical grid points. If the effective sampling rate is lower than the preset sampling rate threshold, random supplementary sampling is performed within the effective area of ​​the sampling mask, and the obtained random sampling points are merged with the effective sampling points to form the base sample point set.

[0010] According to some embodiments of this application, the step of geometrically fitting the base sample point set to obtain a reference surface geometric model includes: For each sample point in the set of basal sample points, the depth value and the depth distance between the depth value and the midpoint of the candidate basal layer interval are calculated, and the basal probability of the corresponding sample point belonging to the basal layer is determined based on the depth distance. Extract a local window centered on the sample point, calculate the depth standard deviation and depth range within the local window, and determine the local uncertainty of the corresponding sample point based on the depth standard deviation and depth range; Based on the basis probability, the local uncertainty, and the preset inlier probability, the comprehensive fitting weight of the sample points is calculated; Based on the comprehensive fitting weights of each sample point, the reference surface geometric model is obtained by performing weighted geometric fitting on the base sample point set through a preset robust loss function.

[0011] According to some embodiments of this application, the step of performing height calibration processing on the initial depth map using the reference surface geometric model to obtain a target calibrated depth map includes: Calculate the fitting residuals of each sample point in the base sample point set to the geometric model of the reference surface, and statistically analyze the proportion of inliers that meet the preset error allowable range based on the fitting residuals. Based on the fitting residuals and the proportion of inliers, calculate the confidence evaluation index of the reference surface geometric model of the current fitting output; If the confidence evaluation index meets the preset compliance conditions, the initial depth map is subjected to pixel-by-pixel height compensation using the reference surface geometric model, so as to translate the reference surface of the initial depth map to the preset sea level height, and the compensated depth map is used as the target calibration depth map. If the confidence evaluation index does not meet the preset compliance conditions, a lossless rollback mechanism is triggered, an alarm message is output, and the initial depth map is used as the target calibration depth map.

[0012] Secondly, embodiments of this application provide a depth map reference surface calibration device, comprising: The acquisition module is used to acquire the initial depth map of the target object; The feature statistics module is used to respectively count the first depth distribution features of the initial depth map in the entire map area and the second depth distribution features in the preset border area; The joint analysis module is used to perform joint analysis based on the first depth distribution features and the second depth distribution features to determine candidate basal layer intervals; A mask generation module is used to generate a sampling mask in the initial depth map based on the candidate basal layer interval and the non-basal region information of the target object; The sampling module is used to perform depth data sampling within the area covered by the sampling mask to obtain a set of base sample points; The fitting module is used to perform geometric fitting on the base sample point set to obtain a reference surface geometric model. The calibration module is used to perform height calibration processing on the initial depth map using the reference surface geometric model to obtain a target calibration depth map.

[0013] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the depth map reference surface calibration method as described in any one of the embodiments of the first aspect of this application.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that is executed by a processor to implement the depth map reference surface calibration method as described in any one of the embodiments of the first aspect of this application.

[0015] This application proposes a depth map reference surface calibration method, apparatus, electronic device, and storage medium. The method first acquires an initial depth map of the target object; it then statistically analyzes a first depth distribution feature within the entire map region and a second depth distribution feature within a preset border region; based on the first and second depth distribution features, it performs joint analysis to determine candidate basal layer intervals; based on the candidate basal layer intervals and the non-basal region information of the target object, it generates a sampling mask in the initial depth map; it samples depth data within the area covered by the sampling mask to obtain a basal sample point set; it performs geometric fitting on the basal sample point set to obtain a reference surface geometric model; and it uses the reference surface geometric model to perform height calibration processing on the initial depth map to obtain a target calibration depth map.

[0016] In contrast to the problems in the background art, the depth map reference surface calibration method provided in this application firstly, by statistically analyzing the depth distribution characteristics of the initial depth map in the entire map region and the preset border region, and performing joint analysis to determine the candidate base layer interval, it overcomes the problem that single-region statistics are easily affected by local feature interference, and can more stably lock the depth range of the base under complex working conditions. Secondly, it generates a sampling mask by combining the candidate base layer interval and the non-base region information of the target object, and performs depth data sampling within the mask coverage area, effectively eliminating interference data from non-base regions such as devices, and ensuring the purity of the extracted base sample point set. Finally, it performs geometric fitting on the base sample point set to obtain a reference surface geometric model, and uses this model to perform height calibration processing on the initial depth map, realizing global benchmark reconstruction based on high-quality sample data. The above steps work together, through joint analysis of dual-region features, mask sampling to eliminate interference, and stepwise processing of geometric fitting, reducing the impact of factors such as local occlusion, board warping, and model changes on benchmark extraction, improving the accuracy of zero-height reference surface estimation of the depth map, and thus reducing systematic measurement errors.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a depth map reference surface calibration method provided in an embodiment of this application.

[0019] Figure 2 This is a flowchart of a depth map reference surface calibration method provided in another embodiment of this application.

[0020] Figure 3This is a flowchart of a depth map reference surface calibration method provided in another embodiment of this application.

[0021] Figure 4 This is a flowchart of a depth map reference surface calibration method provided in another embodiment of this application.

[0022] Figure 5 This is a flowchart of a depth map reference surface calibration method provided in another embodiment of this application.

[0023] Figure 6 This is a flowchart of a depth map reference surface calibration method provided in another embodiment of this application.

[0024] Figure 7 This is a schematic diagram of the structure of a depth map reference surface calibration device provided in an embodiment of this application.

[0025] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0027] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0028] In the description of this application, it should be understood that the orientation descriptions, such as up, down, left, right, front, and back, are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0029] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0030] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setting," "installation," and "connection" should be interpreted broadly. Those skilled in the art can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution. Furthermore, the identification of specific steps in the following text does not imply a limitation on the order of steps or execution logic. The execution order and logic between each step should be understood and inferred from the content described in the embodiments.

[0031] With the advancement of industrial manufacturing towards high automation and precision, industrial 3D vision systems (such as 3D Solder Paste Inspection (3D SPI) and 3D Automated Optical Inspection (3D AOI)) have been widely applied in surface mount technology (SMT) and semiconductor precision manufacturing. These systems typically employ 3D reconstruction techniques such as structured light projection or laser triangulation to acquire 3D point clouds or height information of the printed circuit board surface and generate depth maps containing spatial topography. Based on these depth maps, precision measurement tasks such as solder paste volume measurement, pin coplanarity detection, component height evaluation, and defect identification such as cold solder joints and missing solder joints can be performed. Existing substrate correction methods in engineering practice can be mainly categorized into the following two types: The first category is the planar fitting method based on predefined Regions of Interest (ROIs). This method typically involves engineers pre-defining one or more ROIs considered part of the bare board substrate and using depth data from these regions for planar fitting, thereby normalizing the entire depth map. However, this method is highly dependent on manually defined substrate regions. When the exposed area of ​​the bare board is small, or when the predefined ROI is obscured by components, foreign objects, or abnormal operating conditions (such as component offset, flying parts, etc.), the number of effective substrate points available for fitting will be significantly reduced, leading to a shift in the reference plane estimation. Furthermore, for PCBs (printed circuit boards) with local warping or non-planar deformation, a single planar model often fails to accurately describe the true substrate morphology; and when product model changes or positioning errors cause ROI migration mismatch, the system usually needs to readjust parameters and recalibrate the regions, resulting in high maintenance costs.

[0032] The second category is basis extraction methods based on color segmentation. This method first segments the image based on predefined basis color features, then samples depth data from the segmented basis regions and performs plane fitting. This type of method also relies on manually setting color thresholds and segmentation parameters, and its effectiveness is highly sensitive to lighting conditions, surface reflectivity, and product model variations. Furthermore, there is a lack of direct constraints between color information and geometric properties. When the basis and device surface colors are similar, or when complex conditions such as oxidation, contamination, or printed markings exist, color segmentation is prone to false positives and false negatives, leading to incorrect depth sampling and basis estimation, significantly limiting overall generalization ability and robustness.

[0033] In summary, existing methods generally rely on manually defined prior information (ROI or color features) to determine the base region, and their effectiveness depends on the premise that the base region is predictable and stably visible. When faced with complex assembly scenarios, partial occlusion, board warping, and product model changes, it is often difficult to reliably obtain an accurate zero-height reference plane.

[0034] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a depth map reference surface calibration method, apparatus, electronic device, and storage medium, which can improve the accuracy of depth map zero-height reference surface estimation, thereby reducing systematic measurement errors.

[0035] Reference Figure 1 The depth map reference surface calibration method provided in the embodiments of this application may include, but is not limited to, the following steps 101 to 107: Step 101: Obtain the initial depth map of the target object.

[0036] Step 102: Calculate the first depth distribution features of the initial depth map within the entire map area and the second depth distribution features within the preset border area.

[0037] Step 103: Perform joint analysis based on the first depth distribution features and the second depth distribution features to determine candidate basal layer intervals.

[0038] Step 104: Generate a sampling mask in the depth map based on the candidate basal layer intervals and the non-basal region information of the target object.

[0039] Step 105: Perform depth data sampling within the area covered by the sampling mask to obtain the base sample point set.

[0040] Step 106: Perform geometric fitting on the base sample point set to obtain the reference surface geometric model.

[0041] Step 107: Use the reference surface geometric model to perform height calibration on the depth map to obtain the target calibrated depth map.

[0042] In step 101 of some embodiments, in industrial vision inspection scenarios, 3D reconstruction techniques such as structured light and laser triangulation are typically used to scan and measure the target object (e.g., PCB, FPC, packaging substrate, or microelectronic assembly) to obtain its surface 3D point cloud or height information, thereby constructing an initial depth map, denoted as depth map src. This initial depth map reflects the original spatial morphology of the current target object and serves as the data basis for subsequent substrate reference surface calibration and precision measurement.

[0043] In step 102 of some embodiments, considering that the actual processed product surface may have a high density of device arrangement, a single global height feature is often difficult to stably reflect the true bare board state. Therefore, two different spatial range height statistical features are extracted from the acquired initial depth map. The first depth distribution feature is used to characterize the global distribution trend of the entire depth map at each depth level; the second depth feature is limited by preset parameters. Height statistics are performed within the defined border area. The reason for introducing the preset border area is based on an industrial structural a priori principle: in conventional electronic assembly processes, the border area of ​​the target object is more likely to be in an exposed state without being obscured by components, and therefore the data in this area is more likely to contain the actual bare substrate.

[0044] In some embodiments, step 102 may include, but is not limited to, the following steps: Within a preset depth range, calculate the full-map histogram of the initial depth map within the entire map region, which serves as the first depth distribution feature; Calculate the border histogram of the initial depth map within the preset border area, and use it as the second depth distribution feature.

[0045] Wherein, it is assumed that the preset depth range is Iterate through all pixels in the initial depth map within the entire map region, ensuring that all pixels fall within the preset depth range. All pixels within the range are included in the histogram calculation. The frequency of data points within each depth interval is counted to generate a full-image histogram. This full-image histogram objectively represents the macroscopic height distribution of the entire target object in the global depth space.

[0046] Specifically, the region comprised of pixels within k pixels of the image boundary in the initial depth map is defined as the preset border region, where k is an adjustable parameter that can be set according to the actual physical edge size of the target object. This region is limited to pixels within this preset border region and within the aforementioned preset depth range. The pixels within the bounding box are used in histogram calculation to generate a bounding box histogram.

[0047] In step 103 of some embodiments, after obtaining the depth distribution states of the two dimensions mentioned above, the traditional approach of relying solely on finding the global peak as the basis is abandoned. Instead, the first depth distribution feature reflecting the global distribution and the second depth distribution feature containing prior information about the edge bare board are used together as the basis for layer estimation and joint inference is performed. Through the mutual verification of multi-source statistical information, the risk of substrate misjudgment caused by large-area flat components can be effectively overcome, thereby automatically outputting the absolute depth range of the suspected real substrate from the complex morphology, that is, determining this range as the candidate substrate layer interval. This step determines the candidate layer by leveraging the statistical patterns of the data itself, reducing the reliance on manually specifying the region of interest (ROI).

[0048] In some embodiments, step 103 may include, but is not limited to, the following steps: Cluster analysis was performed based on the full-image histogram to determine the full-image depth cluster set, and cluster analysis was performed based on the border histogram to determine the border depth cluster set. Extract the target full-map depth cluster containing the most data points from the full-map depth cluster set; In the set of bounding box depth clusters, find the local bounding box depth cluster whose depth range is lower than the depth of the target full image and whose depth position is the highest among the remaining bounding box depth clusters; The depth intervals corresponding to the local bounding box depth clusters are determined as candidate basal layer intervals.

[0049] In some embodiments, clustering algorithms such as DBScan (density-based spatial clustering algorithm) or KMeans can be used to analyze several consecutive depth clusters from the full image histogram and the bounding box histogram, respectively, thus forming a full image depth cluster set and a bounding box depth cluster set. During this process, each analyzed depth cluster is parameterized by its depth range and the number of points it contains, specifically denoted as... ,in This is the lower limit of the depth of the depth cluster. is the upper limit of the depth of the depth cluster, and n is the number of data points contained in the depth cluster.

[0050] In some embodiments, by traversing the number of points n in each cluster of the full map depth cluster set, the depth cluster with the most points n is selected and denoted as the target full map depth cluster. In industrial measurement scenarios, the target's full-map depth cluster... This typically corresponds to the flat physical structure that occupies the largest area within the field of view, such as a large-area bare board or a large-sized flat component. This step extracts it as the anchor point for subsequent height logic comparison.

[0051] In some embodiments, since the mounted device is typically above the substrate, if the target full-image depth cluster... For large-sized devices, the actual substrate surface must be lower than the device's depth. Therefore, among the several depth clusters analyzed from the bounding box histogram, the cluster with a depth lower than the target full image is selected. The highest cluster is used as a candidate local basis. The specific mathematical decision-making logic is as follows: for a given bounding box depth cluster... If its depth limit Clusters smaller than the target full map depth The lower limit of its depth, and its lower limit of depth Higher than all others that meet the aforementioned "lower than" If the upper limit of the depth of the border depth cluster of the condition is set, then the border depth cluster is locked as a local border depth cluster.

[0052] In some embodiments, after locking the local bounding box depth cluster through the above-described determination logic, the depth range of the cluster is directly extracted, i.e., the range corresponding to the cluster. The candidate basal layer interval is determined and output to the next stage as the screening interval boundary for the subsequent generation of the absolute depth sampling mask.

[0053] Through the above steps, this application embodiment specifically discloses an implementation process of multi-source histogram joint statistics and clustering inference. It uses clustering algorithms to achieve parameterized extraction of deep clusters and automatically eliminates the interference of large-area fixtures or flat devices through interval logic comparison without the need for manual configuration of gold plate samples or fixed ROI, thus achieving robust localization of the real candidate substrate layer.

[0054] Please see Figure 2 In some embodiments, after step 103, steps 201 to 202 may be included, but are not limited to.

[0055] Step 201: Calculate the support ratio of the pixels corresponding to the candidate basal layer interval within the preset border area.

[0056] Step 202: If the support ratio is lower than the preset ratio threshold, the preset noise filtering parameter is increased, and the first depth distribution feature, the second depth distribution feature, and the candidate basal layer interval are recalculated.

[0057] In step 201 of some embodiments, the candidate substrate layer intervals output in the previous step need to be further verified for their effectiveness in physical spatial distribution. Specifically, the number of pixels whose depth values ​​actually fall within the candidate substrate layer interval is counted, and the ratio of this number of pixels to the total number of effective pixels within the preset border area is calculated. This ratio is used as the support ratio, denoted as sample_to_border_ratio. The support ratio is introduced to quantitatively evaluate the data density of the extracted candidate substrate layers. In typical electronic assembly scenarios, real bare substrates usually exhibit a certain degree of continuous coverage in the edge regions. If the calculated support ratio is low, it usually indicates that the currently extracted candidate substrate layers are too sparse, posing a risk of misjudging local flying point noise, small impurities, or edge pins as substrates, meaning that layer estimation may fail.

[0058] In step 202 of some embodiments, when the support ratio is determined to be lower than a preset ratio threshold, it indicates that the current candidate basal layer has a sparsity failure, thereby triggering a backoff re-evaluation mechanism. The noise filtering parameter is specifically a ratio factor (denoted as p) set during the histogram analysis stage. Assuming the total number of data points participating in the histogram statistics is n, before performing deep clustering analysis on the histogram, the frequency of each depth interval needs to be determined, and intervals with frequencies lower than a certain threshold are excluded. The depth range is considered outlier noise and ignored, preventing it from participating in subsequent deep clustering. After increasing the noise filtering parameters, the threshold value for noise is determined. The noise filtering parameter will increase accordingly, resulting in a greater elimination of depth intervals containing low-frequency interference. While the more depth intervals ignored, the purer the data participating in clustering, an excessively high scaling factor may also lead to the erasure of effective non-noise morphological information. Therefore, the increase of this parameter is usually constrained by reasonable empirical limits. After adopting the increased noise filtering parameter, the statistical, clustering, and candidate layer inference processes of the full-image histogram and bounding box histogram are re-executed to eliminate low-frequency stray points that cause misjudgments under stricter noise filtering conditions, thereby re-outputting candidate base layer intervals with a denser distribution and higher support.

[0059] Through the above steps 201 to 202, this embodiment of the application uses the support ratio to quantitatively verify the initially estimated candidate substrate layer. When the data is determined to be too sparse, the noise filtering parameter is dynamically increased to further suppress low-frequency interference data and re-estimate the data. This enables the overall substrate extraction process to have automatic error correction capability. Thus, even when faced with local contamination, high-frequency noise, or complex edge morphology, it can still stably output a reliable candidate substrate reference surface, improving the robustness of the bottom layer data calibration.

[0060] In step 104 of some embodiments, after obtaining the absolute depth range of the suspected substrate, the initial depth map can first be preliminarily screened in terms of height using this candidate substrate layer range. Simultaneously, considering the device features that inevitably exist on the actual target under test, device metadata (such as known device body, pin, pad, and other location information) is introduced into the processing chain as non-substrate region information. By combining the depth range screening results with the known physical location information of the non-substrate regions, pixels in the aforementioned non-substrate regions are automatically masked and removed from the depth map, thereby generating a sampling mask, denoted as calib_mask, to indicate high-confidence substrate regions. This sampling mask strictly limits the operational boundaries of subsequent processes, preventing abnormal height points from participating in the substrate calculation.

[0061] Please see Figure 3 In some embodiments, step 104 may include, but is not limited to, steps 301 to 303.

[0062] Step 301: Extract the pixels whose depth values ​​are located within the candidate basal layer range in the initial depth map to generate the initial mask.

[0063] Step 302: Obtain the device metadata of the target object, determine the device region of the target object based on the device metadata, and take the device region as the non-base region.

[0064] Step 303: Remove pixels belonging to non-base regions from the initial mask to obtain the sampling mask.

[0065] In step 301 of some embodiments, the candidate basal layer intervals output from the previous step, i.e., the upper and lower limits of the absolute depth range, are used to perform pixel-by-pixel numerical comparison and threshold filtering on the initial depth map. Pixels whose depth values ​​fall within the candidate basal layer interval are marked as valid, while pixels at other depths outside the interval are marked as invalid, thereby completing the binarization process of the entire depth map and generating an initial mask representing the suspected basal region. This initial mask initially locks down data points that conform to the basal features in the height dimension of physical space, but at this time, there may still be other non-basal interference objects mixed in within the mask that are at the same height plane as the basal. Therefore, additional semantic information needs to be introduced for secondary filtering.

[0066] In step 302 of some embodiments, the device metadata typically originates from prior design files or process production data in industrial manufacturing, such as CAD coordinate files, Gerber files, pick-and-place files, or BOM (Bill of Materials) tables. This data contains the two-dimensional layout, physical dimensions, and coordinate position information of various components on the target object (such as a bare PCB board). By parsing this device metadata, its spatial position can be mapped and aligned to the pixel coordinate system of the current initial depth map, thereby accurately identifying the two-dimensional contour areas covered by various mounted components, structural parts, or reserved pads. Since these areas clearly belong to additional devices or structures in terms of physical properties, rather than the actual bare board surface, they are uniformly marked and identified as non-substrate areas that need to be isolated.

[0067] In step 303 of some embodiments, after obtaining the non-basal region defined based on semantic prior, a logical space operation is performed between it and the initial mask generated purely based on a height threshold. Specifically, if a suspected basal pixel in the initial mask overlaps with the non-basal region in two-dimensional space, the state of that pixel is removed from the mask. After the above spatial culling operation between masks, the final output sampling mask retains only pure pixels that both numerically match the basal layer height and avoid interference from known devices in two-dimensional semantic space, thus serving as a high-confidence sampling dataset for subsequent spatial plane fitting.

[0068] Through steps 301 to 303 above, this embodiment of the application combines the depth interval threshold with semantic prior based on device metadata, which effectively makes up for the limitations of relying solely on height numerical judgment. It can eliminate pseudo-substrate interference caused by large-area flat components or test fixtures with heights similar to the substrate, so that the final output sampling mask has extremely high data purity, providing a reliable data source for subsequent three-dimensional geometric construction, and further improving the accuracy and stability of substrate feature extraction in complex component mounting scenarios.

[0069] In step 105 of some embodiments, to ensure that the extracted data objectively and evenly covers the real physical substrate, a sampling strategy prioritizing spatial uniformity can be employed, such as gridded sampling. Through this sampling mechanism, a set of discrete data points satisfying spatial distribution requirements is constructed within the effective area covered by the sampling mask, thus obtaining the substrate sample point set. This sample point set eliminates the influence of excessively dense local noise point clouds, providing a reliable data source for subsequently constructing a uniformly stressed geometric model.

[0070] Please see Figure 4 In some embodiments, step 105 may include, but is not limited to, steps 401 to 403.

[0071] Step 401: Within the sampling mask, perform grid sampling according to the preset physical interval to obtain a set of grid sampling points.

[0072] Step 402: Count the number of effective sampling points that fall within the effective area of ​​the sampling mask, and calculate the ratio of the number of effective sampling points to the total number of theoretical grid points to obtain the effective sampling rate.

[0073] Step 403: If the effective sampling rate is lower than the preset sampling rate threshold, random supplementary sampling is performed within the effective area of ​​the sampling mask, and the obtained random sampling points are merged with the effective sampling points as the base sample point set.

[0074] In step 401 of some embodiments, based on the high-confidence candidate substrate mask calib_mask output from the previous step, to avoid non-substrate structures such as devices, pads, and noise points from participating in subsequent substrate plane estimation, subsequent data sampling and geometric fitting are strictly limited to the suspected substrate region of the sampling mask. During data sampling, to reduce computational complexity while ensuring spatial representativeness of the data, grid sampling is performed on data points within the coverage area of ​​the sampling mask at certain intervals (i.e., preset physical intervals) along the X and Y axes of the two-dimensional plane, thereby constructing a preliminary set of grid sampling points. This regularized operation of sampling at certain x and y intervals ensures that the extracted depth data points have good spatial uniformity.

[0075] In step 402 of some embodiments, because in actual industrial environments, the real bare substrate is often covered by dense components or presents large areas of invalid regions, the mesh sampling extracted at fixed intervals may be insufficient, meaning that not all of the initially generated mesh sampling points fall within the effective area of ​​the mask. Therefore, it is necessary to perform a quantitative statistical analysis of the effectiveness of the mesh sampling point set: assuming the total number of theoretical mesh sampling points divided according to the aforementioned preset physical intervals is n, the number of sampling points that actually fall successfully within the effective area of ​​the sampling mask is counted as p, and then the ratio p / n is calculated. This ratio represents the current effective sampling rate. This effective sampling rate reflects the degree to which the current uniform mesh sampling data captures and covers the effective morphology of the actual complex bare substrate.

[0076] In step 403 of some embodiments, during the actual determination, the logic for triggering random supplementary sampling is determined by judging whether the calculated effective sampling rate p / n meets the standard. When the effective sampling rate is lower than the set sampling rate threshold, it indicates that the effective data points obtained by relying solely on fixed-interval grid sampling are too sparse. At this time, a dynamic data compensation mechanism is triggered, and random sampling is performed directly within the effective area of ​​the sampling mask to obtain an additional number of random pixels. Subsequently, these supplemented random sampling points are combined with the p effective sampling points that fall within the effective area to form the final base sample point set used for spatial fitting calculation.

[0077] Through steps 401 to 403 above, this embodiment of the application utilizes the effective sampling rate p / n as a quantitative evaluation index. When the effective region of the substrate is wide and continuous, the uniformity of data distribution is ensured by setting grid sampling with x and y intervals. When there are large invalid regions in the substrate region, resulting in insufficient grid sampling, random supplementary sampling within the effective region can be triggered in a timely manner. This closed-loop control adaptive sampling strategy not only effectively avoids interference from noise points such as devices and pads by relying on a high-confidence sampling mask (calib_mask), but also takes into account the spatial representativeness and computational efficiency of data sampling, providing data support for subsequent high-robust substrate plane or surface fitting.

[0078] In step 106 of some embodiments, based on the measurement requirements of the actual engineering project and the degree of deformation of the target object, the reference surface geometric model can be a basic planar model, whose expression can be characterized as follows: When faced with non-rigid deformation scenarios such as localized plate warping, fixture deformation, or non-uniform tilting, this model can also be extended to geometric expressions such as quadratic surfaces, segmented planes, or spline surfaces. Through geometric fitting, discrete suspected base samples are transformed into continuous, global reference surface geometric models.

[0079] Please see Figure 5In some embodiments, step 106 may include, but is not limited to, steps 501 to 504.

[0080] Step 501: For each sample point in the base sample point set, calculate the depth value and the depth distance between the depth value and the midpoint of the candidate base layer interval, and determine the base probability of the corresponding sample point belonging to the base based on the depth distance.

[0081] Step 502: Extract a local window centered on the sample point, calculate the depth standard deviation and depth range within the local window, and determine the local uncertainty of the corresponding sample point based on the depth standard deviation and depth range.

[0082] Step 503: Calculate the comprehensive fitting weight of the sample points based on the basis probability, local uncertainty, and preset interior point probability.

[0083] Step 504: Based on the comprehensive fitting weight of each sample point, the base sample point set is subjected to weighted geometric fitting through a preset robust loss function to obtain the reference surface geometric model.

[0084] In step 501 of some embodiments, the specific quantization calculation logic is as follows: assuming the coordinates of the currently extracted sample point in the image space coordinate system are (x, y), and its corresponding actual measured depth value is z(x, y); simultaneously, the midpoint of the depth of the candidate basal layer interval estimated in the previous step is denoted as... The depth range of the candidate basal layer interval is denoted as r. First, the interval assignment of the sample point is determined. If the depth value z(x,y) of the sample point falls outside the candidate basal layer interval, its basal probability is directly assigned. The value is assigned to 0. If the depth value is within the range, a linear mapping calculation is further performed based on the depth distance, and the corresponding mathematical expression is:

[0085] The probability calculation formula intuitively represents an engineering prior: the closer the depth value of the sample point is to the center of the candidate basal layer, the higher its reliability in representing the real basal layer, and the greater the corresponding basal probability.

[0086] In step 502 of some embodiments, in order to measure the stability of a single sample point in the local spatial topography, a local window centered on the current sample point coordinates (x, y) is extracted, denoted as... For all depth data within this local window, calculate their depth standard deviation, denoted as . The maximum depth value within the window is denoted as... And the minimum depth value, denoted as The difference between the maximum and minimum depth values ​​reflects the local depth range. Based on the above statistics, the local uncertainty of this sample point is further calculated, denoted as U(x,y), and the specific mathematical formula is as follows:

[0087] in, This is a preset minimum constant used to prevent anomalies in the denominator when the depth range is zero due to extremely flat local areas. This local uncertainty index reflects the severity of topographic fluctuations in the target local area. The greater the fluctuation, the higher the probability that the point is located at the edge of the device or is subject to noise interference, and the greater the uncertainty of its use as a reference plane.

[0088] In step 503 of some embodiments, after obtaining the basis probability reflecting the degree of proximity of absolute depth and the local uncertainty reflecting the local flatness of space, the preset interior point probability reflecting the geometric consistency during the model iteration process is combined and denoted as... A comprehensive evaluation is then performed on the sample point. The calculation logic for the comprehensive fitting weight w(x,y) of the current sample point can be defined as follows:

[0089] Based on this calculation method, a high-weight, high-quality basal sample point should simultaneously satisfy three characteristic conditions: its depth is close to the center of the basal layer ( Large), conforming to the current iteration's geometric model trend ( Large), and the local topography is smooth ( Small).

[0090] In step 504 of some embodiments, the comprehensive fitting weight w(x,y) of each sample point calculated above is used as a data reliability constraint, and a preset robust loss function is introduced. In the base sample point set The solution is minimized on the above. Let the parametric model of the plane or surface to be estimated be... The goal of weighted geometric fitting is to find the parameter set that minimizes the global weighted error. Its mathematical expression model is as follows:

[0091] During this fitting process, the equation terms This represents the residual between the actual depth of the sample and the estimated depth of the model. By reducing the weights of samples with high uncertainty or low basis probability, and leveraging the outlier-resistant properties of the robust loss function, the final parameter set is obtained through iterative solving. This constitutes the reference surface geometric model.

[0092] Through steps 501 to 504 above, this embodiment of the application quantifies the basis probability of each sample point in the absolute depth distribution and the local uncertainty in the spatial neighborhood, and constructs a comprehensive fitting weight based on this. Combined with iterative optimization of the robust loss function, it can effectively suppress the pulling effect of local small device fragments, edge undulations, and environmental noise on the plane estimation in complex and ever-changing industrial production scenarios, thereby outputting a reference surface geometric model that conforms to the actual bare plate morphology with high accuracy and robustness.

[0093] In step 107 of some embodiments, after fitting a continuous reference surface geometric model, the initial depth map of the original input is subjected to overall benchmark correction based on the model, and uniformly positioned to a preset sea level (i.e., the absolute zero reference surface). Specifically, assuming that the position of a pixel in the initial depth map is (x, y), its original depth value is z(x, y), and the base depth value estimated by the reference surface geometric model at the same position is... The preset sea level height is Then, through mathematical transformation rules The calibrated depth value was calculated. The target calibration depth map output after the above calibration process eliminates the reference offset deviation introduced by physical deformation and pose tilt, so that subsequent operations such as solder paste volume estimation and device height comparison can be based on a unified and reliable reference.

[0094] It should be noted that the sea level in this application refers to the zero-height absolute reference plane uniformly calibrated within the system, which is a custom benchmark under the computer vision spatial coordinate system, rather than altitude in the geographical sense.

[0095] Please see Figure 6 In some embodiments, step 107 may include, but is not limited to, steps 601 to 604.

[0096] Step 601: Calculate the fitting residuals of each sample point in the base sample point set to the reference surface geometric model, and statistically determine the proportion of in-points that meet the preset error allowable range based on the fitting residuals.

[0097] Step 602: Based on the fitting residuals and the proportion of inliers, calculate the confidence evaluation index of the reference surface geometric model of the current fitting output.

[0098] Step 603: If the confidence evaluation index meets the preset criteria, then the reference surface geometric model is used to perform pixel-by-pixel height compensation on the initial depth map to translate the reference surface of the initial depth map to the preset sea level height, and the compensated depth map is used as the target calibration depth map.

[0099] Step 604: If the confidence evaluation index does not meet the preset compliance conditions, the lossless rollback mechanism is triggered, an alarm message is output, and the initial depth map is used as the target calibration depth map.

[0100] In step 601 of some embodiments, for the base sample point set (i.e., the aforementioned set involved in fitting) For any sample point in the array, obtain its actual measured depth value z(x,y) at the current coordinates (x,y), and calculate its estimate of the reference surface geometric model obtained by fitting it with the previous step. The absolute spatial deviation between the data points is defined as the fitting residual for that sample point. After obtaining the fitting residuals of all data within the sample point set, these residual values ​​are compared one by one with a preset inlier determination tolerance threshold. The number of sample points whose fitting residuals fall within this allowable range is counted, and the percentage of this effective number relative to the total number of points in the base sample point set is calculated, thus obtaining the inlier ratio. This inlier ratio directly reflects what proportion of the extracted suspected base samples substantially conforms to the currently calculated reference geometric plane or surface model distribution.

[0101] In step 602 of some embodiments, the consistency of geometric features in the global space is comprehensively considered, and the aforementioned fitting residual statistics reflecting the overall degree of deviation and the proportion of inliers reflecting the model coverage are jointly evaluated and calculated. Generally, the higher the proportion of inliers and the smaller and more concentrated the distribution of fitting residuals, the more realistically and accurately the reference surface geometric model output by the current iteration can reproduce the physical spatial morphology of the bare substrate, thus giving the geometric model a higher confidence evaluation index.

[0102] In step 603 of some embodiments, when the logic determines that the confidence evaluation index of the current model reaches or exceeds a preset safety threshold, it indicates that the fitted reference surface geometric model... It is highly reliable. At this point, using this model as a reference, correction processing is performed on the entire initial depth map. Based on the application of techniques in this field, the mathematical expression for pixel-by-pixel height compensation can be specifically defined as:

[0103] in, This indicates the target calibration depth output by the current pixel after compensation and correction. This represents the estimated depth of the fitted model in the corresponding coordinate system. The set constant is the preset sea level height (a unified zero reference plane). By executing this compensation equation, the low-frequency curvature trend introduced by PCB board warping, fixture deformation under stress, or non-uniform tilting over a large field of view can be effectively eliminated, translating and flattening the entire image's base to a unified sea level. The output map after compensation processing serves as the target calibration depth map, ensuring that all subsequent height threshold determinations, volume estimations, and cross-board comparisons can be performed on a pure and unbiased zero reference.

[0104] In step 604 of some embodiments, if the calculated confidence evaluation index fails to meet the standard, it means that the currently captured base sample has been subjected to severe abnormal interference, such as severe deformation of the entire board, extremely fragmented board surface occlusion, or large-area data loss within the field of view, resulting in a significant risk of distortion in the fitted geometric plane or curved surface. To avoid introducing compensation distortion caused by the erroneous model, which would lead to secondary errors in subsequent measurements, the translation operation of the spatial reference plane is stopped at this time, triggering the system's lossless rollback mechanism. While sending an alarm message to the external production monitoring terminal to indicate that the current board has a calibration abnormality, the initial depth map without deformation compensation is directly output as the target calibration depth map, thereby ensuring the continuity and stability of the industrial production line's production and testing process.

[0105] Through steps 101 to 107 above, this embodiment of the application, without heavily relying on predefined base ROI regions and color segmentation, utilizes the joint analysis of full-map depth distribution and edge structure priors, combined with non-base region information to adaptively generate sampling masks. This enables the automatic acquisition and estimation of the true zero-height reference surface under real industrial noise conditions such as complex assembly scenarios, partial occlusion, and board warping. This effectively solves the systematic measurement deviation introduced by improper reference surface calibration, and improves the consistency of depth map measurement and the flexibility of production line changeover.

[0106] Please see Figure 7 This application also provides a depth map reference surface calibration device, which can implement the above-mentioned depth map reference surface calibration method, including: The acquisition module is used to acquire the initial depth map of the target object; The feature statistics module is used to separately count the first depth distribution features of the initial depth map within the entire map area and the second depth distribution features within the preset border area. The joint analysis module is used to perform joint analysis based on the first depth distribution features and the second depth distribution features to determine candidate basal layer intervals; The mask generation module is used to generate a sampling mask in the initial depth map based on the candidate basal layer interval and the non-basal region information of the target object; The sampling module is used to sample depth data within the area covered by the sampling mask to obtain a set of base sample points; The fitting module is used to perform geometric fitting on the base sample point set to obtain the reference surface geometric model; The calibration module is used to perform height calibration on the initial depth map using a reference surface geometric model to obtain the target calibration depth map.

[0107] In contrast to the problems in the background art, the depth map reference surface calibration method provided in this application firstly, by statistically analyzing the depth distribution characteristics of the initial depth map in the entire map region and the preset border region, and performing joint analysis to determine the candidate base layer interval, it overcomes the problem that single-region statistics are easily affected by local feature interference, and can more stably lock the depth range of the base under complex working conditions. Secondly, it generates a sampling mask by combining the candidate base layer interval and the non-base region information of the target object, and performs depth data sampling within the mask coverage area, effectively eliminating interference data from non-base regions such as devices, and ensuring the purity of the extracted base sample point set. Finally, it performs geometric fitting on the base sample point set to obtain a reference surface geometric model, and uses this model to perform height calibration processing on the initial depth map, realizing global benchmark reconstruction based on high-quality sample data. The above steps work together, through joint analysis of dual-region features, mask sampling to eliminate interference, and stepwise processing of geometric fitting, reducing the impact of factors such as local occlusion, board warping, and model changes on benchmark extraction, improving the accuracy of zero-height reference surface estimation of the depth map, and thus reducing systematic measurement errors.

[0108] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in a memory, and the processor executes the at least one program to implement the depth map reference surface calibration method described above. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0109] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 802 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and called and executed by the processor 801 using the depth map reference plane calibration method of the embodiments of this application. The 803 input / output interface is used to implement information input and output. The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804); The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0110] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described depth map reference surface calibration method.

[0111] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0113] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0116] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or the indirect coupling or communication connection between the apparatus or units may be electrical, mechanical, or other forms.

[0119] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for calibrating a depth map reference surface, characterized in that, include: Obtain the initial depth map of the target object; The method involves calculating the first depth distribution feature of the initial depth map within the entire map region and the second depth distribution feature within the preset border region, including: within a preset depth range, calculating the full map histogram of the initial depth map within the entire map region, which is used as the first depth distribution feature; Calculate the border histogram of the initial depth map within the preset border area, and use it as the second depth distribution feature; The joint analysis based on the first depth distribution feature and the second depth distribution feature determines candidate basal layer intervals, including: performing cluster analysis based on the full-image histogram to determine a full-image depth cluster set, and performing cluster analysis based on the border histogram to determine a border depth cluster set; extracting the target full-image depth cluster containing the most data points from the full-image depth cluster set; finding, within the border depth cluster set, a local border depth cluster with a depth interval lower than the target full-image depth cluster and the highest depth position among the remaining border depth clusters; and determining the depth interval corresponding to the local border depth cluster as the candidate basal layer interval. Based on the candidate basal layer interval and the non-basal region information of the target object, a sampling mask is generated in the initial depth map, including: extracting pixels whose depth values ​​are located within the candidate basal layer interval in the initial depth map to generate an initial mask; obtaining device metadata of the target object, determining the device location region of the target object based on the device metadata, and using the device location region as the non-basal region; removing pixels belonging to the non-basal region from the initial mask to obtain the sampling mask; Depth data is sampled within the area covered by the sampling mask to obtain a base sample point set; The geometric fitting of the basis sample point set to obtain a reference surface geometric model includes: for each sample point in the basis sample point set, calculating the depth distance between the depth value and the midpoint of the candidate basal layer interval, and determining the basis probability of the corresponding sample point belonging to the basis based on the depth distance; extracting a local window centered on the sample point, calculating the depth standard deviation and depth range within the local window, and determining the local uncertainty of the corresponding sample point based on the depth standard deviation and depth range; calculating the comprehensive fitting weight of the sample point based on the basis probability, the local uncertainty, and the preset inlier probability; and performing weighted geometric fitting on the basis sample point set using a preset robust loss function based on the comprehensive fitting weight of each sample point to obtain the reference surface geometric model. The initial depth map is height-calibrated using the reference surface geometric model to obtain the target calibration depth map.

2. The method according to claim 1, characterized in that, After determining candidate basal layer intervals through joint analysis based on the first depth distribution features and the second depth distribution features, the method further includes: Calculate the support ratio of the pixels corresponding to the candidate base layer interval within the preset border area; If the support ratio is lower than a preset ratio threshold, the preset noise filtering parameter is increased, and the first depth distribution feature, the second depth distribution feature, and the candidate basal layer interval are recalculated; wherein, the increased noise filtering parameter is used to remove depth data with lower frequency during histogram statistics and cluster analysis.

3. The method according to claim 1, characterized in that, The process of performing depth data sampling within the area covered by the sampling mask to obtain a base sample point set includes: Within the sampling mask, grid sampling is performed at preset physical intervals to obtain a set of grid sampling points; The effective sampling rate is obtained by counting the number of valid sampling points that fall within the effective area of ​​the sampling mask and calculating the ratio of the number of valid sampling points to the total number of theoretical grid points. If the effective sampling rate is lower than the preset sampling rate threshold, random supplementary sampling is performed within the effective area of ​​the sampling mask, and the obtained random sampling points are merged with the effective sampling points to form the base sample point set.

4. The method according to claim 1, characterized in that, The step of performing height calibration processing on the initial depth map using the reference surface geometric model to obtain the target calibrated depth map includes: Calculate the fitting residuals of each sample point in the base sample point set to the geometric model of the reference surface, and statistically analyze the proportion of inliers that meet the preset error allowable range based on the fitting residuals. Based on the fitting residuals and the proportion of inliers, calculate the confidence evaluation index of the reference surface geometric model of the current fitting output; If the confidence evaluation index meets the preset compliance conditions, the initial depth map is subjected to pixel-by-pixel height compensation using the reference surface geometric model, so as to translate the reference surface of the initial depth map to the preset sea level height, and the compensated depth map is used as the target calibration depth map. If the confidence evaluation index does not meet the preset compliance conditions, a lossless rollback mechanism is triggered, an alarm message is output, and the initial depth map is used as the target calibration depth map.

5. A depth map reference surface calibration device, characterized in that, include: The acquisition module is used to acquire the initial depth map of the target object; The feature statistics module is used to respectively count the first depth distribution feature of the initial depth map in the entire map area and the second depth distribution feature in the preset border area, including: within the preset depth range, calculating the full map histogram of the initial depth map in the entire map area as the first depth distribution feature; Calculate the border histogram of the initial depth map within the preset border area, and use it as the second depth distribution feature; The joint analysis module is used to perform joint analysis based on the first depth distribution feature and the second depth distribution feature to determine candidate basal layer intervals, including: performing cluster analysis based on the full image histogram to determine a full image depth cluster set, and performing cluster analysis based on the border histogram to determine a border depth cluster set; extracting the target full image depth cluster containing the most data points from the full image depth cluster set; finding, within the border depth cluster set, a local border depth cluster with a depth interval lower than the target full image depth cluster and the highest depth position among the remaining border depth clusters; and determining the depth interval corresponding to the local border depth cluster as the candidate basal layer interval. A mask generation module is used to generate a sampling mask in the initial depth map based on the candidate basal layer interval and the non-basal region information of the target object, including: extracting pixels whose depth values ​​are located within the candidate basal layer interval in the initial depth map to generate an initial mask; obtaining device metadata of the target object, determining the device location region of the target object based on the device metadata, and taking the device location region as the non-basal region; removing pixels belonging to the non-basal region from the initial mask to obtain the sampling mask; The sampling module is used to perform depth data sampling within the area covered by the sampling mask to obtain a set of base sample points; The fitting module is used to perform geometric fitting on the base sample point set to obtain a reference surface geometric model. This includes: for each sample point in the base sample point set, calculating the depth distance between the depth value and the midpoint of the candidate base layer interval, and determining the base probability of the corresponding sample point belonging to the base based on the depth distance; extracting a local window centered on the sample point, calculating the depth standard deviation and depth range within the local window, and determining the local uncertainty of the corresponding sample point based on the depth standard deviation and depth range; calculating the comprehensive fitting weight of the sample point based on the base probability, the local uncertainty, and a preset inlier probability; and performing weighted geometric fitting on the base sample point set using a preset robust loss function based on the comprehensive fitting weight of each sample point to obtain the reference surface geometric model. The calibration module is used to perform height calibration processing on the initial depth map using the reference surface geometric model to obtain a target calibration depth map.

6. An electronic device, characterized in that, include: The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the depth map reference surface calibration method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the depth map reference surface calibration method as described in any one of claims 1 to 4.

Images