Depth sensor anomaly detection method and system, storage medium and electronic equipment

Abstract

The invention discloses a depth sensor anomaly detection method and system, a storage medium and electronic equipment. The method comprises the steps that an absolute depth value collected by a depth sensor and a relative depth value output by a monocular depth estimation model are acquired; global sorting is carried out on the absolute depth value and the relative depth value, a sorting difference matrix between the absolute depth value and the relative depth value is further calculated, and a sorting inconsistency score vector is obtained; obtaining a first-stage detection threshold value by adopting a fused adaptive threshold value strategy based on the sorting inconsistency score vector, and forming an initial anomaly set; for each point in the non-abnormal point set, constructing a comprehensive similarity scoring matrix after fusion; and accumulating the comprehensive similarity scoring matrix according to rows to obtain a second-stage scoring vector, and determining a final refined abnormal point set. According to the invention, by introducing the relative depth information generated based on the monocular image, the robustness and environmental adaptability of the depth sensing system can be significantly improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision depth perception, and more particularly, relates to a depth sensor anomaly detection method and system, a storage medium and an electronic device. BACKGROUND

[0002] With the rapid development of three-dimensional perception technologies such as augmented reality, autonomous driving, and robot navigation, devices have higher requirements for the acquisition and processing of depth information in the environment. As a key to spatial understanding, depth perception is widely used in core tasks such as object detection, path planning, scene reconstruction, and pose estimation, directly affecting the intelligent decision-making and interaction performance of the system. Current mainstream depth perception hardware includes structured light, stereo vision, laser radar, and depth sensors based on time-of-flight (ToF). In actual applications, various types of depth sensors generally have some degree of measurement anomalies. Typical manifestations include missing depth values, edge distortion, background penetration, numerical jumps, or incorrect measurements on mirror, transparent, black, and other surfaces. Notably, such problems not only occur in low-cost sensors, but even industrial-grade devices are difficult to completely avoid in complex environments or on surfaces of special materials. These measurement anomalies not only affect the accuracy of depth information itself, but also can cause deviations in the system's understanding of the environment, thereby affecting the overall stability and task execution effect. Therefore, abnormal measurement by depth sensors has become a common problem, and effective detection and suppression mechanisms are urgently needed to address it.

[0003] Most existing anomaly detection methods are designed for specific types of scenes, especially focusing on mirror, transparent, and other special surfaces. Such methods usually rely on color image information, identify object materials or regional features, indirectly infer the abnormal positions in the depth map, and perform correction. For example, by extracting mirror regions or transparent boundaries, and combining surface normal information, the corresponding depth values are interpolated or compensated. These methods mostly rely on multiple image processing modules running in coordination, with complex processes and a strong dependence on the semantic recognition ability of specific objects or materials.

[0004] Since such methods are essentially designed for task customization, their scope of application is limited, and they are difficult to generalize to other types of anomalies. In addition, such methods often process dense and high-resolution depth maps, with high computational cost and resource consumption, making them unsuitable for deployment in terminal devices or real-time systems, and difficult to apply to sparse depth data processing. Therefore, existing methods have obvious deficiencies in efficiency and adaptability, making it difficult to meet the actual needs of depth anomaly detection in multiple scenes and multiple device environments. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a depth sensor anomaly detection method, system, storage medium and electronic device. By introducing relative depth information generated based on monocular images, the robustness and environmental adaptability of the depth perception system can be significantly improved, providing more reliable depth input data for typical applications such as three-dimensional reconstruction, augmented reality, robot navigation, and autonomous driving, solving the problem that the depth sensor in the existing depth perception system has measurement anomalies and is difficult to accurately identify and process, especially in resource-constrained scenarios, without introducing learning parameters, how to efficiently, universally and low-costly screen out and correct error values in sparse or dense depth data.

[0006] In order to achieve the above-mentioned purpose, according to the first aspect of the present application, a depth sensor anomaly detection method is provided, comprising:

[0007] S100: obtaining absolute depth values collected by a depth sensor and relative depth values output by a monocular depth estimation model, and synchronously obtaining pixel coordinate information corresponding to each point;

[0008] S200: globally sorting the absolute depth values and the relative depth values respectively, obtaining corresponding ranking vectors, and further calculating a sorting difference matrix between the two, to obtain a sorting inconsistency score vector;

[0009] S300: obtaining a first-stage detection threshold based on the sorting inconsistency score vector using a fusion adaptive threshold strategy, and determining points with significant inconsistent sorting as abnormal points based on the first-stage detection threshold to form an initial abnormal set;

[0010] S400: for each point in the non-abnormal point set, calculating three normalized difference indicators of absolute depth, relative depth and image space position between the point and all abnormal points in the set respectively, and fusing to construct a comprehensive similarity score matrix;

[0011] S500: accumulating the comprehensive similarity score matrix by row to obtain a second-stage score vector, and combining a maximum threshold and a limit addition proportion mechanism to determine a final refined abnormal point set, thereby effectively identifying abnormal patterns in the depth measurement process.

[0012] Further, in S200, the ranking vector is:

[0013]

[0014] Wherein: D represents an input depth vector, sgn is a sign function, and N represents the number of collected points.

[0015] Further, in S200, the sorting difference matrix is:

[0016]

[0017] Further, in S200, the ranking inconsistency score vector comprises:

[0018] Based on the ranking difference matrix, a ranking inconsistency matrix I is obtained:

[0019]

[0020] wherein δ is a smoothing factor, Q abs is the ranking difference matrix of absolute depth values, Q rel is the ranking difference matrix of relative depth values.

[0021] Further, in S200, the ranking inconsistency score vector is:

[0022]

[0023] Further, in S300, the first stage detection threshold is obtained by weighted fusion of two candidate thresholds, comprising:

[0024] S301: a first candidate threshold t otsu is obtained by Otsu algorithm, satisfying the principle of maximum inter-class variance;

[0025] S302: a second candidate threshold t norm is calculated based on statistical distribution:

[0026] t norm = Mean(s) + λ · Var(s)

[0027] wherein λ is an empirical weight, and s is the ranking inconsistency score vector;

[0028] S303: a weighted coefficient w is determined by a Spearman rank correlation coefficient r s and a Sigmoid function:

[0029] w = sigmoid(k(r s -u)),

[0030] t coarse = w · t norm + (1-w) · t otsu

[0031] wherein k is a slope adjustment factor, and u is a center position.

[0032] Further, the Spearman rank correlation coefficient r s is:

[0033]

[0034] Where: R abs With R rel There are two ranking vectors.

[0035] Furthermore, in S400, the comprehensive similarity scoring matrix consists of the following three similarity indicators:

[0036]

[0037] Where: d represents absolute depth, r represents relative depth, p represents pixel coordinates, and ε is a constant to prevent the denominator from being zero.

[0038] Furthermore, in S400, the comprehensive similarity scoring matrix is:

[0039]

[0040] Where α, β, and γ are exponential decay coefficients.

[0041] Furthermore, in S500, the second-stage scoring vector is:

[0042]

[0043] Where: N c N represents the number of outliers. r S represents the number of remaining points. ij This is a comprehensive similarity scoring matrix. Furthermore, S500 also includes a second-stage anomaly detection threshold:

[0044] t fine =max(T) Otsu (s′),TopK(s′,N c / 4))

[0045] Where: t otsu The first candidate threshold is defined as TopK, which represents the Nth threshold selected after sorting all scores in s′. c / 4 is a large value.

[0046] Furthermore, the method is applied to dense depth map scenarios, and its efficiency is improved through the following optimization mechanism:

[0047] S601: Downsample the input high-resolution depth map to 1 / n resolution, perform ranking consistency scoring at low resolution, and the mean within the block represents the overall features;

[0048] S602: In the region similarity scoring stage, the regular arrangement structure of blocks is used to perform similarity scoring on pixels within each abnormal block and pixels in its neighboring normal blocks, and a closed-form formula is used instead of Otsu's calculation:

[0049]

[0050] wherein: sigma is an adjustment factor, epsilon is a small constant.

[0051] According to a second aspect of the present application, there is provided a depth sensor anomaly detection system, comprising:

[0052] a data collection module configured to obtain absolute depth values collected by the depth sensor and relative depth values output by the monocular depth estimation model, and to obtain pixel coordinate information corresponding to each point synchronously;

[0053] a ranking inconsistency scoring module configured to perform global ranking on the absolute depth values and the relative depth values respectively, to obtain corresponding ranking vectors, and to further calculate a ranking difference matrix between the two, to obtain a ranking inconsistency scoring vector;

[0054] an initial anomaly identification module configured to obtain a first-stage detection threshold based on the ranking inconsistency scoring vector using a fused adaptive threshold strategy, and to determine points with significant ranking inconsistency as anomaly points based on the first-stage detection threshold, to form an initial anomaly set;

[0055] a comprehensive similarity scoring module configured to calculate three kinds of normalized difference indicators of absolute depth, relative depth and image space position between each point in the non-anomaly point set and all anomaly points in the set respectively, and to fuse the three kinds of normalized difference indicators to construct a comprehensive similarity scoring matrix;

[0056] a final anomaly identification module configured to obtain a second-stage scoring vector by accumulating the comprehensive similarity scoring matrix row by row, and to determine a final refined anomaly point set by combining a maximum threshold and a limit addition proportion mechanism, so as to effectively identify anomaly patterns in the depth measurement process.

[0057] According to a third aspect of the present application, there is provided a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the depth sensor anomaly detection method.

[0058] According to a fourth aspect of the present application, there is provided an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the depth sensor anomaly detection method when executing the computer program.

[0059] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects compared with the prior art:

[0060] 1.The method of the present application, by introducing the relative depth information generated by monocular image, combining with the absolute depth information provided by depth sensor, constructing two-stage abnormal scoring mechanism of ordering consistency and regional similarity, effectively identifying abnormal patterns such as random error, boundary jump, structure occlusion in depth measurement process, solving the problem that the depth sensor in the existing depth perception system cannot accurately identify and process due to measurement abnormalities, especially in resource-limited scenarios, without introducing learning parameters, how to efficiently, universally and low-costly screen and correct the error values in sparse or dense depth data.

[0061] 2.The method of the present application, proposes an adaptive threshold fusion strategy based on Spearman rank correlation coefficient guidance, combines Otsu maximum between-class variance method and statistical model to construct dual candidate threshold, and dynamically adjusts its fusion weight according to ordering consistency, realizing robust adaptation to different abnormal proportions and data distribution.

[0062] 3.The method of the present application, proposes a low-overhead abnormality detection method based on image blocks, with a two-stage mechanism compatible with sparse patterns, significantly reducing the dimension and computational complexity of the scoring matrix, ensuring the accuracy and efficiency of abnormal point determination.

[0063] 4.The method of the present application, without introducing learning parameters, realizes universal, fast and low-cost screening of abnormal data output by various depth sensors, provides more stable and reliable depth input for three-dimensional reconstruction, augmented reality, autonomous driving, robot perception and other scenarios, significantly improves the overall perception ability and environmental adaptability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The depth sensor abnormality detection method flowchart of the embodiment of the present application;

[0065] Figure 2 The device structure schematic diagram suitable for depth abnormality detection of the embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0067] As Figure 1As shown, this embodiment proposes a depth sensor anomaly detection method. The core idea is to use the relative depth generated by the monocular depth estimation model as auxiliary information, combined with the absolute depth output of the sensor, to construct a two-stage scoring mechanism based on ranking consistency and region similarity, so as to achieve accurate identification and screening of outliers in sparse or dense depth points, including:

[0068] S100: Acquires the absolute depth value collected by the depth sensor and the relative depth value output by the monocular depth estimation model, and simultaneously obtains the pixel coordinate information corresponding to each point;

[0069] S200: The absolute depth value and the relative depth value are globally sorted respectively to obtain the corresponding ranking vectors, and the sorting difference matrix between the two is further calculated to obtain the sorting inconsistency score vector.

[0070] S300: Based on the ranking inconsistency score vector, an adaptive threshold strategy is adopted to obtain the first-stage detection threshold, and points with significant ranking inconsistency are identified as anomalies, forming an initial anomaly set.

[0071] S400: For each point in the set of non-abnormal points, calculate three normalized difference indices between it and all abnormal points in the set in terms of absolute depth, relative depth and image spatial location, and then fuse them to construct a comprehensive similarity score matrix.

[0072] S500: The second-stage scoring vector is obtained by accumulating the row-wise comprehensive similarity scoring matrix, and the final refined set of outliers is determined by combining the maximum threshold and the limit on the proportion of new additions, thereby effectively identifying abnormal patterns in the depth measurement process.

[0073] like Figure 2 As shown, the method of this invention introduces relative depth information generated from monocular images and combines it with absolute depth information provided by a depth sensor to construct a two-stage anomaly scoring mechanism based on ranking consistency and region similarity. This effectively identifies anomalies such as random errors, boundary jumps, and structural occlusion during depth measurement, solving the problem of inaccurate identification and processing of measurement anomalies in existing depth sensing systems. Particularly in resource-constrained scenarios, it addresses how to efficiently, universally, and cost-effectively filter and correct erroneous values ​​in sparse or dense depth data without introducing learning parameters. It requires no learning parameters or training process and is suitable for depth data preprocessing on resource-constrained devices.

[0074] Example 1

[0075] like Figure 1 As shown in the embodiment of the present invention, a method for calculating deep anomaly scores based on ranking consistency and region similarity is provided, specifically including:

[0076] 1. Data input and definition: Input data includes: absolute depth values obtained by depth sensor: D abs ∈ N ; relative depth values output by monocular depth estimation model: D rel ∈ N ; corresponding image coordinate information: P∈ N×2 The target output of this embodiment is a binary anomaly mask M∈0,1 N , where M i =1 indicates that the ith point is determined to be an outlier, and M i =0 indicates a normal point.

[0077] 2. First stage: Anomaly score calculation based on ordering consistency

[0078] This stage aims to evaluate the consistency of the ordering position of each depth point in two depth domains (absolute and relative), and use it as a coarse-grained anomaly determination basis.

[0079] Ranking vector calculation: For any depth vector D=[D1,D2,...,D N ], calculate its ranking vector R∈N, where each element is defined as follows:

[0080]

[0081] Where sgn(·) is the sign function. By the above calculation, the absolute depth ranking vector R abs and the relative depth ranking vector R rel are obtained respectively.

[0082] Ordering relationship matrix construction: According to the above ranking vector, construct the normalized ordering difference matrix Q∈ N×N , which is calculated as follows:

[0083]

[0084] The above matrix is applied to the absolute depth and relative depth domains, denoted as Q abs and Q rel , respectively, and N represents the number of collected points.

[0085] Ordering consistency difference calculation: To measure the difference between the two ordering relationships, calculate the ordering consistency difference matrix I:

[0086]

[0087] Where δ=0.05 is a smoothing factor used to suppress small differences in flat areas.

[0088] Finally, for each depth point, its rank inconsistency score is calculated:

[0089]

[0090] This score reflects the degree of rank difference of the point in two depth domains, the higher the value, the greater the rank inconsistency of the point, thus more likely to be an outlier point.

[0091] 3. Second stage: abnormal boundary refinement based on regional similarity

[0092] Considering that the first stage may miss detection in edge regions or regions where structures are not obvious, this stage introduces a regional similarity scoring mechanism to further refine the outlier detection results. Based on the scoring results of the first stage, set the outlier point set C and the normal candidate point set R, where |C| = N c , |R| = N r , and satisfy N c +N r =N.

[0093] Similarity index calculation: for each outlier point x i ∈C and candidate normal point y j ∈R, calculate the absolute depth difference degree v abs , the relative depth difference degree v rel , and the image space pixel distance w space Three types of normalized similarity indexes:

[0094]

[0095] Where d i and r i represent the absolute and relative depth of point x i , p i represents the image coordinates, and ε is a minimum value set to prevent the denominator from being zero (e.g., 0.001).

[0096] Comprehensive similarity score matrix calculation: the three types of indexes are fused by exponential weighting to construct the final similarity score matrix Its definition is:

[0097]

[0098] Where α, β, γ are hyperparameters that control the weights of different indexes, usually set to 10.

[0099] Candidate point score and final determination: for each candidate point y j ∈R, accumulate the similarity scores with all outlier points to get the final similarity score:

[0100]

[0101] The higher the score, the stronger the similarity between the candidate point and the anomaly point, which should be further marked as a potential anomaly. By combining the ranking consistency and regional similarity two-stage scoring, this embodiment can efficiently identify various types of anomaly points, adapt to the error characteristics of depth sensors in different scenarios, and have good explainability and deployment convenience without introducing model learning, suitable for embedding in various depth perception systems as a pre-processing or anomaly screening module.

[0102] Embodiment 2

[0103] In addition, in another embodiment of the present application, an adaptive threshold calculation method based on ranking consistency correlation is provided for accurately determining whether a depth point is abnormal, specifically comprising:

[0104] This method considers three threshold strategies: the first is the Otsu method based on inter-class variance, the second is the normal distribution strategy based on statistical mean and variance, and the third is the fusion method based on the adjustment weight of the Spearman rank correlation coefficient of ranking consistency. Through the combination of the three, robust threshold selection can be achieved in the case of uncertain abnormal point proportion change and abnormal distribution characteristics.

[0105] 1. Otsu threshold calculation

[0106] Otsu method is an unsupervised image segmentation technique, which is suitable for dividing the score value (such as anomaly score) into "normal" and "abnormal" two categories. Its core idea is: among all possible threshold values, select the threshold value that can maximize the inter-class variance. The specific steps are as follows:

[0107] • Assume that the score set is s1, s2,..., s N , sorted in ascending order;

[0108] • For each possible threshold t, divide the data into two categories: C0 = s i ∣s i ≤t and C1 = s i ∣s i >t;

[0109] • Calculate the probabilities p0, p1, means μ0, μ1 of the two categories respectively;

[0110] • Calculate the inter-class variance:

[0111] • Select the maximum among all possible t as the final Otsu threshold, denoted as t otsu .

[0112] This method works well when the proportion of outliers is significant and their distribution is concentrated, but when the proportion of outliers is extremely low, it may be misjudged as a large area of ​​outliers due to the insignificant inter-class variance.

[0113] 2. Statistical threshold of normal distribution

[0114] To compensate for Otsu's limitations under extreme distributions, this embodiment introduces a judgment threshold based on statistical regularities. Let the score set be s1, s2, ..., s N The statistical threshold is calculated using the following formula:

[0115] t norm =Mean(s) + λ·Var(s)

[0116] in The coefficient λ controls the threshold strength, and empirically it is set to 2.5, which approximately conforms to the 3σ principle.

[0117] When the score distribution is close to a unimodal skewed distribution, this strategy can effectively identify outliers far from the mean and avoid large-scale false alarms.

[0118] 3. Spearman Ranking Consistency Index

[0119] Spearman ranking correlation coefficient r s It is a nonparametric statistical indicator used to measure the monotonic correlation between two variables. Its numerical range is [-1, 1], where r... s =1 indicates perfect positive correlation (consistent ranking); r s =-1 indicates a perfect negative correlation; r s =0 indicates that the ranking is irrelevant.

[0120] In this invention, the absolute depth ranking vector R is calculated. abs With relative depth ranking vector R rel The Spearman correlation coefficient is used as a global measure of the consistency of the rankings of the two entities. Its calculation formula is as follows:

[0121] Sort the two sets of depth values ​​to obtain the ranking vector R. abs ,R rel ∈ N ;

[0122] • Calculate the sum of squared differences between the two rankings:

[0123] The final Spearman coefficient is:

[0124] The closer the value is to 1, the more consistent the ordering relationship between the two depth sources is, the more reliable the overall observation is, and the lower the error probability is; otherwise, the inconsistency degree is large and the error risk is high.

[0125] 4. Fusion strategy

[0126] Considering the advantages and disadvantages of Otsu and statistical threshold in different scenarios, the two are weighted and fused through the Spearman coefficient to construct the final adaptive threshold:

[0127] w = sigmoid(k(r s -u)),

[0128] t coarse = w·t norm +(1-w)·t otsu .

[0129] Wherein: k represents the coefficient of adjusting the slope, controls the switching speed, and the empirical value is k=50; u is the intermediate threshold, which is set as u=0.8, which means that when r s =0.8, each of the two accounts for 50%; is a standard S-shaped function.

[0130] The formula is more inclined to the conservative statistical threshold when the ordering consistency (r s ) is strong; and is more inclined to the Otsu threshold sensitive to anomalies when the ordering is chaotic (r s ). Through the adaptive fusion strategy, the threshold weight can be automatically switched according to different input scenarios, effectively avoiding over-detection or missing detection problems. The method of the present application proposes an adaptive threshold fusion strategy guided by the Spearman rank correlation coefficient, constructs dual candidate thresholds combining the Otsu maximum between-class variance method and the statistical model, and dynamically adjusts the fusion weight according to the ordering consistency, realizing robust adaptation to different abnormal proportions and data distribution. The method of the present application realizes general, fast and low-cost screening of abnormal data output by various depth sensors without introducing learning parameters, provides more stable and reliable depth input for three-dimensional reconstruction, augmented reality, autonomous driving, robot perception and other scenes, and significantly improves the overall perception ability and environmental adaptability of the system.

[0131] Embodiment 3

[0132] In another embodiment of the present application, a block-level anomaly detection mechanism suitable for dense depth maps is provided, considering that the output of high-precision depth sensors (such as industrial-grade ToF or laser radar) is usually a high-resolution depth map, and the calculation complexity of constructing a full matrix directly on all pixels is O(H 2 W 2), extremely high, not suitable for resource-constrained devices. To this end, the embodiment proposes a low-overhead anomaly detection method based on image blocks, with a two-stage mechanism compatible with sparse mode.

[0133] 1. Image block division and depth downsampling

[0134] The original depth map is divided into blocks of size n x n (e.g. n = 8, 16), and the average pooling method is used to obtain low-resolution absolute and relative depth maps. After downsampling, the overall computational complexity is reduced to:

[0135]

[0136] 2. Block-level ordering consistency score

[0137] On the low-resolution image, the ordering consistency score is calculated for each image block, which is completely consistent with the sparse mode, and the suspicious blocks are preliminarily screened.

[0138] 3. Pixel-level similarity refinement

[0139] For image blocks marked as abnormal, further local similarity scoring is performed on all pixels within them. Unlike the sparse mode, the spatial term w space is not used here, but the difference between the pixel and its k neighboring normal block pixels in absolute and relative depth is calculated:

[0140] S = exp(-αV rel )·exp(-βV abs )

[0141] where k = 8 and each block size is n x n. The total complexity is:

[0142] O(n 4 NM) → O(Nn 2 ·kn 2 )

[0143] Significantly better than full-image calculation.

[0144] 4. Fast threshold calculation

[0145] A closed-form expression is used to avoid repeated histogram calculation and improve efficiency:

[0146]

[0147] where σ = 0.005 and ε = 0.001. This formula adaptively adjusts the threshold with variance changes to control the risk of false positives. The method of the invention proposes a low-overhead anomaly detection method based on image blocks, with a two-stage mechanism compatible with sparse mode, significantly reducing the dimension of the scoring matrix and the computational complexity, ensuring the accuracy and efficiency of anomaly point determination.

[0148] Embodiment 4

[0149] The embodiment of the present application also provides a depth sensor anomaly detection system, comprising:

[0150] a data acquisition module, configured to acquire absolute depth values collected by a depth sensor and relative depth values output by a monocular depth estimation model, and to synchronously obtain pixel coordinate information corresponding to each point;

[0151] a ranking inconsistency scoring module, configured to respectively perform global ranking on the absolute depth values and the relative depth values, to respectively obtain corresponding ranking vectors, and to further calculate a ranking difference matrix between the two, to obtain a ranking inconsistency scoring vector;

[0152] an initial anomaly identification module, configured to obtain a first-stage detection threshold based on the ranking inconsistency scoring vector by using a fused adaptive threshold strategy, and to determine points with significant ranking inconsistency as anomaly points according to the first-stage detection threshold, to form an initial anomaly set;

[0153] a comprehensive similarity scoring module, configured to respectively calculate, for each point in the non-anomaly point set, three normalized difference indexes of the point and all anomaly points in the set in terms of absolute depth, relative depth and image space position, to fuse the three indexes, and to construct a comprehensive similarity scoring matrix;

[0154] a final anomaly identification module, configured to obtain a second-stage scoring vector by performing row accumulation on the comprehensive similarity scoring matrix, and to determine a final refined anomaly point set by combining a maximum threshold and a limit addition proportion mechanism, so as to effectively identify anomaly patterns in a depth measurement process.

[0155] In addition, the anomaly detection system for the actual depth sensor in the embodiment comprises one or more processors and a memory, the processor and the memory can be connected through a bus or other means, Figure 2 for example, by taking the bus connection as an example. The memory as a non-volatile computer readable storage medium can be used to store non-volatile software programs and non-volatile computer executable programs, such as the efficient stereo matching method for large disparity scenes in the above embodiment. The processor executes the non-volatile software programs and instructions stored in the memory, thereby executing the anomaly detection method for the actual depth sensor.

[0156] The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory can optionally include a memory that is remotely located with respect to the processor, which can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0157] The program instructions / modules are stored in the memory, and when executed by the one or more processors, perform the above-described embodiments of the anomaly detection method for a real depth sensor, for example, perform the above-described embodiments of the anomaly detection method for a real depth sensor. Figure 1 Each of the steps shown.

[0158] The embodiments of the present application also provide a computer storage medium, which has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement the anomaly detection method for a real depth sensor provided by the embodiments of the present application.

[0159] Those skilled in the art can easily understand that the above description is only the preferred embodiments of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method of depth sensor anomaly detection, the method comprising: Comprising: S100: obtaining absolute depth values collected by a depth sensor and relative depth values output by a monocular depth estimation model, and synchronously obtaining pixel coordinate information corresponding to each point; S200: globally sorting the absolute depth values and the relative depth values respectively, obtaining corresponding ranking vectors, and further calculating a sorting difference matrix between the two, to obtain a sorting inconsistency score vector; S300: obtaining a first-stage detection threshold based on the sorting inconsistency score vector using a fused adaptive threshold strategy, and determining points with significant sorting inconsistency as abnormal points based on the first-stage detection threshold to form an initial abnormal set; S400: for each point in the non-abnormal point set, calculating three normalized difference indicators of the point in absolute depth, relative depth, and image space position with all abnormal points in the set, and fusing to construct a comprehensive similarity score matrix; S500: accumulating the comprehensive similarity score matrix by row to obtain a second-stage score vector, and combining a maximum threshold with a limit addition proportion mechanism to determine a final refined abnormal point set, thereby effectively identifying abnormal patterns in the depth measurement process.

2. The method of claim 1, wherein, In S200, the ranking vector is: Wherein: D represents an input depth vector, sgn is a sign function, and N represents the number of collected points.

3. The method of claim 2, wherein, In S200, the sorting difference matrix is:

4. The method of claim 3, wherein, In S200, the sorting inconsistency score vector comprises: The sorting inconsistency matrix I is obtained based on the sorting difference matrix: where δ is a smoothing factor, Q abs is a sorted difference matrix of absolute depth values, Q rel is a sorted difference matrix of relative depth values.

5. The method of claim 4, wherein, In S200, the sorting inconsistency score vector is:

6. The method of claim 1-5, wherein, In S300, the first-stage detection threshold is obtained by weighted fusion of two candidate thresholds, comprising: S301: the first candidate threshold t otsu Obtained by the Otsu algorithm, satisfying the maximum inter-class variance principle; S302: second candidate threshold t norm Based on statistical distribution calculation: t norm = Mean(s) + λ · Var(s) Wherein, λ is an empirical weight, and s is a sorting inconsistency score vector; S303: The weighting coefficient w is determined by the Spearman rank correlation coefficient r s with the sigmoid function w = sigmoid(k(r s -u)), t coarse = w · t norm + (1 - w) · t otsu Wherein: k is a slope adjustment factor, and u is a center position.

7. The method of claim 6, wherein, The Spearman rank correlation coefficient r s is: wherein: R abs and R rel are two ranking vectors.

8. The method of claim 1-5, wherein, In S400, the comprehensive similarity score matrix is composed of the following three similarity indicators: Wherein: d represents absolute depth, r is relative depth, p is pixel coordinates, and ε is a constant to prevent the denominator from being zero.

9. The method of claim 8, wherein, In S400, the comprehensive similarity score matrix is: Wherein: α, β, and γ are exponential decay coefficients.

10. The method of claim 1-5, wherein, In S500, the second-stage score vector is: Where: N c is the number of outliers, N r is the number of remaining points, S ij is the combined similarity score matrix.

11. The method of claim 10, wherein, The second-stage abnormality determination threshold in S500 is: t fine = max(T Otsu (s′), TopK(s′, N c / 4)) wherein: t otsu is the first candidate threshold, and TopK represents the value of the Nth largest score in s′ after sorting all scores in s′. c / 4.

12. The method of claim 1-5, wherein, The method is applied to a dense depth map scene, and efficiency is improved through the following optimization mechanism: S601: downsample the input high-resolution depth map to 1 / n resolution, and perform sorting consistency scoring at low resolution, with the block mean representing the overall feature; S602: in the regional similarity scoring stage, use the regular arrangement structure of the block to perform similarity scoring on each abnormal block pixel and its neighborhood normal block pixel, and replace the Otsu calculation with a closed-form formula: Where: σ is an adjustment factor, and ε is a minimum constant.

13. A depth sensor anomaly detection system, characterized in that it comprises: a data acquisition module for obtaining absolute depth values collected by a depth sensor and relative depth values output by a monocular depth estimation model, and synchronously obtaining pixel coordinate information corresponding to each point; a ranking inconsistency scoring module configured to globally rank the absolute depth values and the relative depth values respectively to obtain corresponding ranking vectors, and further to calculate a ranking difference matrix between the two vectors to obtain a ranking inconsistency scoring vector; an initial anomaly identification module configured to obtain a first-stage detection threshold based on the ranking inconsistency scoring vector using a fused adaptive threshold strategy, and to determine points with significant ranking inconsistency as anomaly points based on the first-stage detection threshold to form an initial anomaly set; a comprehensive similarity scoring module configured to calculate three kinds of normalized difference indicators of each point in the non-anomaly point set with all anomaly points in the set in terms of absolute depth, relative depth and image spatial position respectively, and to fuse the indicators to construct a comprehensive similarity scoring matrix; a final anomaly identification module configured to obtain a second-stage scoring vector by accumulating the comprehensive similarity scoring matrix by row, and to determine a final refined anomaly point set by combining a maximum threshold with a limit addition proportion mechanism, so as to effectively identify anomaly patterns in the depth measurement process.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the depth sensor anomaly detection method according to any one of claims 1-12.

15. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the depth sensor anomaly detection method according to any one of claims 1-12.

Images