Fault detection method and device of DOE, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a fault detection method and device of a DOE, electronic equipment and a storage medium, and relate to the technical field of face recognition. The method comprises: acquiring a first depth map set based on a natural scene photographed by a DOE to be detected, and determining first depth distribution information of the first depth map set; determining a first difference between the first depth distribution information and at least one second depth distribution information acquired in advance; and if all the first differences do not exceed a preset difference range, determining that the DOE to be detected is normal. The embodiments of the present application do not need to use additional hardware for auxiliary detection, have a lower detection cost, and only need a few depth maps of the DOE to be detected, so as to accurately judge whether the DOE is faulty, have a higher detection efficiency, and have a good popularization and application value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition, in particular, the present application relates to a DOE fault detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] Diffractive optical elements (DOE) are widely used in optical devices due to their high diffraction efficiency and outstanding phase modulation performance. For the core component of structured light depth camera, structured light projection module, the performance of DOE directly affects the quality of the projection pattern, and further affects the depth imaging effect of the depth camera.

[0003] Due to the internal structure size of the DOE reaching microns or even nanometers, errors in the manufacturing process or damage in the transportation process will directly affect the performance of the DOE, and after assembly, it will cause poor consistency of the projection module, and in some cases, it will also cause strong zero-order diffraction laser hazards.

[0004] The related art often needs to detect the DOE with the help of additional equipment, which not only affects the detection efficiency, but also increases the use cost. SUMMARY

[0005] The embodiments of the present application provide a DOE fault detection method, device, electronic equipment and storage medium which overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, a DOE fault detection method is provided, the method comprising:

[0007] obtaining a first set of depth maps based on a natural scene photographed by a DOE to be detected, determining first depth distribution information of the first set of depth maps;

[0008] determining a first difference between the first depth distribution information and each of at least one second depth distribution information obtained in advance;

[0009] if all first differences do not exceed a preset difference range, determining that the DOE to be detected is normal;

[0010] wherein each second depth distribution information is depth distribution information of a second set of depth maps obtained based on a normal DOE photographing a calibration scene, and the depth distribution information is used to represent the number of sampling points of the corresponding set of depth maps in each depth value interval.

[0011] In one possible implementation, the fault detection method further comprises:

[0012] if at least one first difference exceeds the preset difference range, determining that the DOE to be detected is abnormal.

[0013] In one possible implementation, the preset difference range is obtained based on the second difference between at least one second depth distribution information and at least one pre-acquired third depth distribution information;

[0014] Each third depth distribution information is the depth distribution information of a third depth atlas obtained based on the abnormal DOE shooting calibration scene.

[0015] In one possible implementation, the fault detection method also includes determining a preset range of differences:

[0016] Determine the second difference between any pairwise second depth distribution information and any pairwise third depth distribution information;

[0017] The minimum value among all the second differences is used as the initial upper limit of the preset difference range.

[0018] One possible implementation uses the minimum value among all second differences as the initial upper limit of a preset difference range, followed by:

[0019] Acquire at least one fourth depth map set based on natural scenes captured by normal DOE, and determine the fourth depth distribution information for each fourth depth map set;

[0020] Determine the third difference between at least one second depth distribution information and at least one fourth distribution information pairwise;

[0021] Based on the distribution of all third differences, adjust the initial upper limit of the preset difference range and determine the lower limit of the preset difference range.

[0022] One possible implementation involves adjusting the initial upper limit of the preset difference range and determining the lower limit of the preset difference range based on the distribution of the third difference, including:

[0023] The minimum value among all third differences is used as the lower limit of the preset difference threshold.

[0024] The smaller of the maximum value in all third differences and the initial upper limit value is used as the upper limit value of the preset difference threshold.

[0025] One possible implementation involves adjusting the initial upper limit of the preset difference range and determining the lower limit of the preset difference range based on the distribution of the third difference, including:

[0026] Based on the distribution of the third difference, determine the expected value and variance σ of the distribution, and determine the distribution interval based on the 3σ principle;

[0027] The smaller of the initial upper limit value and the upper limit value of the distribution interval is used as the upper limit value of the preset difference range;

[0028] The lower limit of the distribution interval is used as the lower limit of the preset difference range.

[0029] One possible implementation involves calculating the difference between any two depth distribution information, including:

[0030] Construct a difference matrix, which includes the difference in the number of sampling points of the two depth distribution information in each depth value interval;

[0031] The difference between the two depth distribution information can be obtained by using the transpose of the difference matrix and the inverse of the covariance matrix of the difference matrix.

[0032] In one possible implementation, in the calibration scenario, the DOE and the calibration board are at a preset distance and a preset angle, the calibration board is a flat plate, and there are multiple preset distances.

[0033] Secondly, a fault detection device for DOE is provided, comprising:

[0034] The first depth distribution information acquisition module is used to acquire a first depth map set based on the natural scene captured by the DOE to be detected, and to determine the first depth distribution information of the first depth map set.

[0035] The first difference acquisition module is used to determine the first difference between the first depth distribution information and at least one pre-acquired second depth distribution information;

[0036] The judgment module is used to determine that the DOE to be detected is normal if all the first differences do not exceed the preset difference range;

[0037] Each second depth distribution information is the depth distribution information of a second depth map obtained based on a normal DOE shooting calibration scene. The depth distribution information is used to represent the number of sampling points of the corresponding depth map in each depth value interval.

[0038] In one possible implementation, the judgment module is further configured to: determine that the DOE to be detected is abnormal if at least one first difference exceeds a preset difference range.

[0039] In one possible implementation, the preset difference range is obtained based on the second difference between at least one second depth distribution information and at least one pre-acquired third depth distribution information;

[0040] Each third depth distribution information is the depth distribution information of a third depth atlas obtained based on the abnormal DOE shooting calibration scene.

[0041] In one possible implementation, the fault detection device further includes a difference range acquisition module, which further includes:

[0042] The second difference acquisition submodule is used to determine the second difference between any pair of second depth distribution information and any third depth distribution information.

[0043] The range determination submodule is used to take the minimum value among all the second differences as the initial upper limit value of the preset difference range.

[0044] In one possible implementation, the difference range acquisition module also includes:

[0045] The fourth depth distribution information acquisition module is used to acquire at least one fourth depth map set based on natural scenes captured by normal DOE, and to determine the fourth depth distribution information of each fourth depth map set.

[0046] The third difference determination module is used to determine the third difference between at least one second depth distribution information and at least one fourth distribution information;

[0047] The upper and lower limit adjustment module is used to adjust the initial upper limit value of the preset difference range and determine the lower limit value of the preset difference range based on the distribution of all third differences.

[0048] In one possible implementation, the upper and lower limit adjustment module includes:

[0049] The first lower limit determination module is used to take the minimum value among all the third differences as the lower limit value of the preset difference threshold.

[0050] The first upper limit determination module is used to take the smaller of the maximum value in all third differences and the initial upper limit value as the upper limit value of the preset difference threshold.

[0051] In one possible implementation, the upper and lower limit adjustment module includes:

[0052] The distribution interval determination module is used to determine the expected value and variance σ of the distribution based on the distribution of the third difference, and to determine the distribution interval based on the 3σ principle;

[0053] The second upper limit determination module is used to take the smaller value between the initial upper limit value and the upper limit value of the distribution interval as the upper limit value of the preset difference range;

[0054] The second lower limit determination module is used to take the lower limit value of the distribution interval as the lower limit value of the preset difference range.

[0055] In one possible implementation, the fault detection device further includes a difference calculation module for calculating the difference between any two depth distribution information, the difference calculation module including:

[0056] The matrix construction unit is used to construct the difference matrix, which includes the difference in the number of sampling points of two depth distribution information in each depth value interval;

[0057] The difference calculation unit is used to obtain the difference between two depth distribution information based on the transpose of the difference matrix and the inverse of the covariance matrix of the difference matrix.

[0058] In one possible implementation, in the calibration scenario, the DOE and the calibration board are at a preset distance and a preset angle, the calibration board is a flat plate, and there are multiple preset distances.

[0059] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method provided in the first aspect.

[0060] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect.

[0061] Fifthly, embodiments of this application provide a computer program that includes computer instructions stored in a computer-readable storage medium. When a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps that implement the method provided in the first aspect.

[0062] The DOE fault detection method, apparatus, electronic device, and storage medium provided in this application embodiment acquire a first depth map set based on a natural scene captured by the DOE to be detected, determine the first depth distribution information of the first depth map set, and determine the first difference between each pair of the first depth distribution information and at least one pre-acquired second depth distribution information. Each second depth distribution information is the depth distribution information of a second depth map set acquired based on a normal DOE capturing a calibration scene. The preset difference range can characterize the reasonable range of difference between the depth distribution information of a non-abnormal DOE and the depth distribution information of a normal DOE. Therefore, if all first differences do not exceed the preset difference range, the DOE to be detected is determined to be normal. This solution does not require additional hardware to assist in detection, has a low detection cost, and only requires a few depth maps of the DOE to be detected to accurately determine whether the DOE is faulty, resulting in higher detection efficiency and good application value. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0064] Figure 1 This is a schematic diagram of the structure of a structured light camera provided in an embodiment of this application;

[0065] Figure 2 This is a flowchart illustrating the fault detection method according to an embodiment of this application;

[0066] Figure 3 This is a schematic diagram illustrating an application scenario of an embodiment of this application;

[0067] Figure 4 This is a flowchart illustrating the fault detection method according to an embodiment of this application;

[0068] Figure 5 This is a schematic diagram showing the distribution of depth values ​​obtained from each sampling point in an embodiment of this application;

[0069] Figure 6 This is a schematic diagram of the DOE calibration system according to an embodiment of this application;

[0070] Figure 7 This is a schematic diagram illustrating the process of obtaining the second depth atlas according to an embodiment of this application;

[0071] Figure 8 This is a schematic diagram illustrating the process of obtaining the third depth atlas according to an embodiment of this application;

[0072] Figure 9 This is a flowchart illustrating a fault detection method according to another embodiment of this application;

[0073] Figure 10 This is a schematic diagram illustrating the setting of a preset difference threshold based on the second difference and the third difference in an embodiment of this application.

[0074] Figure 11 A flowchart illustrating the process of determining a preset difference range for embodiments of this application;

[0075] Figure 12 This is a flowchart illustrating the fault detection method of this application in a specific application scenario;

[0076] Figure 13 This is a schematic diagram of the structure of a fault detection device provided in an embodiment of this application;

[0077] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0078] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0079] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0080] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0081] First, let's introduce and explain several terms used in this application:

[0082] Depth map: A depth map is an image or image channel that contains information about the distance from the surface of a scene object to the viewpoint. Each pixel in the depth map represents the vertical distance between the depth camera plane and the plane of the object being photographed, usually represented by 16 bits, in millimeters. In facial recognition payment, it is generally used for: liveness detection and assisting in comparative recognition.

[0083] Optimization: Select a set of color images, depth images, and infrared images that meet the prerequisites for liveness detection and comparison recognition algorithms. Optimization is achieved by selecting the color image based on face angle, face size, face centering, and clarity; the infrared image based on brightness; and the depth image based on completeness.

[0084] Liveness detection: a method to determine whether the person using facial recognition is a real person, a photo, or a head model. Generally, depth maps are used to determine if it is a photo, and infrared brightness maps are used to determine if it is a silicone head model. More in-depth and specific details are the core secrets of each company's facial recognition technology.

[0085] Comparison and Recognition: The comparison and recognition of which user is being identified through facial recognition generally involves extracting five special similarities from a color image and then using a depth image to assist in comparing the three-dimensional similarities of the five special similarities. The more in-depth and specific details are the core secrets of each company's facial recognition technology.

[0086] Mahalanobis distance: Mahalanobis distance represents the covariance distance of data. It is an efficient method for calculating the similarity between two unknown sample sets.

[0087] DOE: Diffractive optical elements (DOEs) are typically constructed using micro- and nano-etching processes to create two-dimensional diffraction units. Each diffraction unit can have a specific morphology, refractive index, etc., allowing for precise control of the laser wavefront phase distribution. When the laser passes through each diffraction unit, it undergoes diffraction and interference occurs at a certain distance (usually infinity or the focal plane of a lens), resulting in a specific intensity distribution.

[0088] The 3σ criterion, also known as the Raida criterion, first assumes that a set of test data contains only random errors. It then calculates the standard deviation and determines an interval based on a certain probability. Errors exceeding this interval are considered gross errors, not random errors, and data containing such errors should be discarded. The 3σ criterion is as follows:

[0089] The probability that the value is distributed in (μ-σ,μ+σ) is 0.6827;

[0090] The probability that the value is distributed in (μ-2σ,μ+2σ) is 0.9545;

[0091] The probability that the value is distributed in (μ-3σ,μ+3σ) is 0.9973;

[0092] It can be assumed that the values ​​are almost entirely concentrated in the range of (μ-3σ, μ+3σ), and the probability of them falling outside this range is less than 0.3%.

[0093] The fault detection method, apparatus, electronic device, and computer-readable storage medium for DOE provided in this application are intended to solve the above-mentioned technical problems of the prior art.

[0094] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0095] Please see Figure 1 The figure exemplifies the structural schematic diagram of a structured light camera, which includes:

[0096] Light source 101 is used to emit a beam of light;

[0097] Lens 102 is used to converge the light beam emitted by light source 101 and project a parallel light beam to DOE 103;

[0098] DOE103 receives and splits the parallel beam, and projects the diffracted beam onto the acquisition object 104;

[0099] The detector 105 collects the diffracted beam reflected by the object 104 to construct a pattern of structured light. Since the pattern will be deformed due to the shape of the object, the depth value of each point of the object is calculated by using the trigonometric principle based on the degree of deformation.

[0100] A normal DOE projects a diffracted beam that is divergent relative to a parallel beam. This means that the DOE illuminates a large area of ​​light onto the object being sampled. As a result, each coordinate point in the depth map obtained by the detector has corresponding depth information. However, when the DOE is damaged, the diffracted beam does not diverge completely or even not at all. This results in only some coordinate points in the middle of the image having depth information in the depth map, while other areas lack depth information. Therefore, based on this characteristic, this application embodiment uses depth map-based detection to determine whether the DOE is faulty.

[0101] Please see Figure 2 The example illustrates a flowchart of a fault detection method according to an embodiment of this application. This application can acquire depth maps captured by a DOE to be detected and depth maps captured by a normal DOE. The depth distribution information of the two depth maps is statistically analyzed. The difference between the two depth distribution information is calculated—specifically, it can be represented by the Mahalanobis distance. If the difference is greater than a preset threshold range, then the DOE is considered to be damaged. It should be understood that the threshold range can be obtained based on the difference in depth distribution information between the depth maps captured by a normal DOE and the depth maps captured by an abnormal DOE.

[0102] Please see Figure 3This example illustrates an application scenario to which the embodiments of this application can be adapted. In this scenario, the application includes a monitoring device 201 with face recognition based on a structured light camera, a fault detection device 202, a fault detection device 203, a display device 204, and a printer 205. The monitoring device 201 is used to acquire images of industrial products, such as circuit boards, to obtain the original product image of the industrial product to be inspected. Both the fault detection devices 202 and 203 can use the fault detection method provided in this embodiment to detect whether a DOE (Design for Objects) in the monitoring device is faulty. The monitoring device 201 is communicatively connected to the fault detection devices 202 and 203, allowing the depth map captured by the monitoring device 201 to be sent to the fault detection devices 202 and 203 for effective identification of the DOE based on the depth map.

[0103] The difference between fault detection device 202 and fault detection device 203 is that fault detection device 202 has a display function. Therefore, the fault detection result determined by fault detection device 202 can be directly presented to the user, while the fault detection result determined by fault detection device 203 can be sent to display device 204 for display. Figure 2 In this device, the fault detection device 203 is communicatively connected to the display device 204 so that the fault detection device 203 can send the fault detection results to the display device 204 and present them to the user in a timely manner.

[0104] Additionally, this application scenario may also include a printer 205, which can print out the fault detection results of the fault detection device 203 for users to view on paper. Of course, the fault detection device 202 can also communicate with the display device 204 and the printer 205. For simplicity, in... Figure 3 Not all of them are shown in the diagram.

[0105] It should be noted that when any embodiment of this application is applied to a specific product or technology, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0106] Please see Figure 4 The figure illustrates a flowchart of a DOE fault detection method according to another embodiment of this application, as shown in the figure, including:

[0107] S101. Obtain the first depth map set based on the natural scene captured by the DOE to be detected, and determine the first depth distribution information of the first depth map set.

[0108] The natural scenarios in this application embodiment can be everyday face recognition scenarios—such as face payment scenarios, access control scenarios, terminal unlocking scenarios, etc. In these scenarios, the distance between the face and the DOE is generally between 25 cm and 50 cm. That is, each depth map in the first depth map set is acquired by the DOE within a distance of 25 cm to 50 cm from the scene object—the face. Each depth map contains information related to the distance from the surface of the scene object to the viewpoint (equivalent to the DOE). It should be emphasized that all depth maps in the first depth map set obtained in this application embodiment are depth maps that have undergone information anonymization processing.

[0109] This application embodiment does not impose a specific limit on the number of depth maps (i.e., the first depth maps) in the first depth map set, but the number does not need to be large, for example, no more than 5. The distances corresponding to each depth map can be the same or different, and the scene objects can be the same or different.

[0110] It should be understood that the first depth distribution information refers to the distribution of depth values ​​across all depth maps in the first depth atlas, such as the number of sampling points at each depth, the number of sampling points in each depth interval, and the distribution position of each depth value in the depth map. This application does not impose specific limitations on these details. For ease of understanding, the following embodiments use the number of sampling points in each depth interval as an example.

[0111] Please see Figure 5 The figure illustrates, by way of example, the distribution of depth values ​​obtained from each sampling point in an embodiment of this application. As shown in the figure, there are 16 grids, representing 16 sampling points. For ease of understanding, the depth value obtained from the corresponding sampling point is marked in each grid. Statistically, the depths of the 16 sampling points are as follows: 2 sampling points have a depth of 25.0 cm, 2 sampling points have a depth of 25.5 cm, 2 sampling points have a depth of 26.0 cm, 3 sampling points have a depth of 26.1 cm, 2 sampling points have a depth of 26.6 cm, 1 sampling point has a depth of 26.9 cm, 2 sampling points have a depth of 27.0 cm, and 2 sampling points have a depth of 27.4 cm.

[0112] By setting three depth intervals, namely (25.0, 26.0], (26.0, 27.0], and (27.0, 28.0], the depth distribution information of the depth map can be determined as follows: there are 6 sampling points in the (25.0, 26.0] interval, 8 sampling points in the (26.0, 27.0] interval, and 2 sampling points in the (27.0, 28.0] interval.

[0113] S102. Determine the first difference between the first depth distribution information and at least one pre-acquired second depth distribution information.

[0114] Before executing step S102, this embodiment of the application requires the prior acquisition of second depth distribution information. The second depth distribution information represents the depth distribution information of the second depth map set. In this embodiment, the second depth map set is a depth map set obtained based on normal DOE (Direct Optical Equipment) shooting of the calibration scene. The distance between the DOE and the scene object in the calibration scene is a number of preset values, such as 25 cm, 26 cm, ..., 50 cm. More importantly, the scene object is not an object with an irregular surface, such as a human face, but a flat plate. With this setting, if the DOE and the calibration plate are perpendicular, the depth values ​​of all sampling points in the depth image are the same; if there is a certain tilt angle between the DOE and the calibration plate, the depth values ​​of all samples in the review image also vary uniformly.

[0115] Please see Figure 6 The figure exemplifies the structural schematic diagram of the DOE calibration system according to an embodiment of this application. As shown, it includes a track 401 with a scale indicating the distance from the calibration plate. A base 402 is slidably mounted on one end of the track 401, and the base 402 can slide on the track to the corresponding scale to acquire a depth map of the response depth value.

[0116] The base 402 is also connected to a servo motor 403. The servo motor can control speed and has very accurate positioning. It can convert voltage signals into torque and speed to drive the controlled object. The rotor speed of the servo motor is controlled by the input signal and can respond quickly. In automatic control systems, it is used as an actuator and has characteristics such as small electromechanical time constant and high linearity. In this embodiment, the servo motor 403 controls the base 402 to slide on the track 401. A multi-degree-of-freedom servo motor 404 is mounted above the base 402. Since this embodiment needs to ensure that the DOE is as perpendicular to the calibration board as possible and does not require a large tilt angle, the servo motor does not need to have too many degrees of freedom. For example, a two-degree-of-freedom servo motor (rotation in the left and right directions and rotation in the up and down directions) can be selected, which can reduce costs and meet calibration requirements.

[0117] The servo motor 404 is fixedly connected to the structured light camera 405, which includes a DOE (Design for Electronics), so that when the servo motor 404 rotates, the structured light camera 405 can also rotate synchronously. A calibration plate 406 is set at the other end of the track 401. In addition, the calibration system also includes a computer 407, which is connected to the servo motor, the servo motor, and the structured light camera, and can control the movement of the servo motor and the servo motor, as well as generate a depth map based on the sampling results of the structured light camera.

[0118] When acquiring the second depth map, the computer controls the servo motor to move to a preset distance according to the preset step size, controls the servo motor to rotate so that the shooting direction of the structured light camera is as perpendicular as possible to the calibration board, and then controls the structured light camera to sample. Multiple samples are taken at each distance, and the sampling results are returned to the computer, which processes them into multiple depth maps.

[0119] It should be understood that, since the distance between the structured light camera and the face in actual face recognition scenarios is generally in the range of 25 to 50 centimeters, the preset distance in this application embodiment can also start from 25 centimeters and gradually increase in preset steps (e.g., 1 centimeter) until it reaches 50 centimeters.

[0120] It should be noted that for a normal DOE, N depth maps are obtained based on each sampling distance. If we assume there are a total of M sampling distances, then M×N depth maps can be obtained. For the M×N depth maps, they can be further divided into P second depth map sets. Each second depth map set includes M×N÷P depth maps (assuming M×N is divisible by P). The depth maps in the second depth map set can be depth maps randomly selected from the M×N depth maps, or they can be several depth maps randomly selected from multiple depth maps at each sampling distance, until M×N÷P depth maps are obtained.

[0121] Please see Figure 7 The figure illustrates, for example, a flowchart of obtaining a second depth map set according to an embodiment of this application. As shown in the figure, firstly, e preset distances (L1, L2, ..., Le) are determined, and M depth maps are collected at each preset distance. Taking distance L1 as an example, the collected depth map is named L1-depth. Figure 1 L1-Depth Figure 2 ...L1-Depth Map M. When constructing each second depth map set, m depth maps are randomly extracted from the M depth maps collected at each preset distance to obtain e×m depth maps to obtain a second depth map set.

[0122] It should be noted that, for ease of understanding, Figure 7 The process involves extracting the first depth map corresponding to different distances to construct the first second depth map set, and extracting the second depth map corresponding to different distances to construct the second second depth map set. However, in actual applications, the number of depth maps extracted for each distance can be random, and which depth map is extracted can also be random.

[0123] To ensure detection accuracy, the second depth map set contains a significantly larger number of depth maps than the first depth map set; for example, the second depth map set can contain over 1000, or even 10000, depth maps. Since the second depth map set is captured by a normal DOE in a calibration scene, the difference obtained by comparing the first and second depth distribution information can, to some extent, reflect the differences between the tested DOE and a normal DOE.

[0124] S103. If all first differences do not exceed the preset difference range, it is determined that the DOE to be tested does not have a fault.

[0125] It should be noted that since there is more than one second depth distribution information in this application embodiment, the first difference between the first depth distribution information and each second depth distribution information can be calculated. The preset difference range in this application embodiment can characterize the reasonable difference range between the depth distribution information of a normal DOE in a natural scene and the depth distribution information in a calibrated scene. Therefore, if the difference between the first depth distribution information and the second depth distribution information of the DOE to be detected does not exceed the preset difference range, then the DOE to be detected can be identified as having a fault. If all the first differences do not exceed the preset difference range, then the DOE to be detected is determined to be without a fault, i.e., normal.

[0126] The DOE fault detection method of this application embodiment acquires a first depth map set based on a natural scene captured by the DOE to be detected, determines the first depth distribution information of the first depth map set, and determines the first difference between each pair of the first depth distribution information and at least one pre-acquired second depth distribution information. Each second depth distribution information is the depth distribution information of a second depth map set acquired based on a normal DOE capturing a calibration scene. The preset difference range can characterize the reasonable range of difference between the depth distribution information of a non-abnormal DOE and the depth distribution information of a normal DOE. Therefore, if all first differences do not exceed the preset difference range, the DOE to be detected is determined to be normal. This solution does not require additional hardware to assist in detection, has a low detection cost, and only requires a few depth maps of the DOE to be detected to accurately determine whether the DOE is faulty, resulting in higher detection efficiency and good application value.

[0127] Based on the above embodiments, as an optional embodiment, the fault detection method further includes: if at least one of the first differences exceeds the preset difference range, then the DOE to be detected is determined to be abnormal.

[0128] In this embodiment of the application, after determining that the DOE is abnormal, an alarm can also be issued to the user or owner of the structured light camera, thereby stopping the application of the structured light camera in a timely manner and ensuring the accuracy and security of the identification.

[0129] Based on the above embodiments, as an optional embodiment, the preset difference range of this application embodiment is obtained based on the second difference between at least one second depth distribution information and at least one pre-acquired third depth distribution information.

[0130] The third depth distribution information represents the depth distribution information of the third depth atlas. The third depth atlas is a depth atlas obtained based on the calibration scene captured by an abnormal DOE. It should be understood that the calibration scene corresponding to the third depth atlas is the same as the calibration scene corresponding to the second depth distribution information. The difference lies in the DOE used.

[0131] The number of abnormal DOEs used in this application embodiment can be multiple. Different abnormal DOEs have different fault types, and each abnormal DOE has at least one fault type. In this way, the depth distribution information of the third depth atlas is the depth distribution information collected by DOEs with various fault types.

[0132] When obtaining the third depth map, similar to obtaining the second depth map, for each depth map, the computer adjusts the servo motor to a preset distance, controls the servo to rotate so that the shooting direction of the structured light camera is as perpendicular as possible to the calibration board, and then controls the structured light camera (the DOE in the structured light camera is a damaged DOE) to sample, and returns the sampling results to the computer, which processes them into a depth map. During the sampling process, sampling can be performed separately for DOEs with different fault types.

[0133] It should be understood that, since the distance between the structured light camera and the face in actual face recognition scenarios is generally in the range of 25 to 50 centimeters, the preset distance in this application embodiment can also start from 25 centimeters and gradually increase in preset steps (e.g., 1 centimeter) until it reaches 50 centimeters.

[0134] For an abnormal DOE, for each fault type, K depth maps can be obtained at each sampling distance. If there are J fault types and M sampling distances, then K×M×J depth maps can be obtained. For the K×M×J depth maps, they can be further divided into Q third depth map sets. Each second depth map set includes K×M×J÷Q depth maps (assuming K×M×J is divisible by Q). The depth maps in the third depth map set can be randomly selected from the K×M×J depth maps, or they can be randomly selected from multiple depth maps corresponding to each fault type and sampling distance, until K×M×J÷Q depth maps are obtained.

[0135] Please see Figure 8The figure illustrates a flowchart of obtaining a third depth map in an embodiment of this application. As shown, firstly, e preset distances (L1, L2, ..., Le) are determined, and f abnormal DOEs (DOE1, DOE2, ..., DOEf) are obtained. The fault types of each abnormal DOE are not exactly the same. "Not exactly the same" means that no two abnormal DOEs have completely identical fault types.

[0136] For each anomalous DOE, N depth maps are collected at each preset distance. Taking DOE1 sampled at distance L1 as an example, the collected depth maps are named D1-L1- Figure 1 D1-L1- Figure 2 ..., D1-L1-Figure N. When constructing each third depth atlas, n depth maps are randomly extracted from the N depth maps collected at each preset distance for each anomalous DOE, resulting in e*f*n depth maps, thus obtaining a third depth atlas.

[0137] It should be noted that, for ease of understanding, Figure 8 The first method extracts the first depth map corresponding to different distances for each fault type to construct the first third depth map set, and extracts the second depth map corresponding to different distances for each fault type to construct the second second depth map set. However, in actual applications, the number of depth maps extracted for each distance and each fault type can be random, and which depth map is extracted can also be random.

[0138] This application embodiment calculates and statistically analyzes the second differences between each pair of second depth distribution information and third depth distribution information to obtain the distribution range of the second differences. For example, if there are two second depth distribution information and two third depth distribution information, then by combining them pairwise, four second differences can be obtained, and the range of values ​​of these four second differences is the distribution range of the second differences.

[0139] Understandably, if the first difference between the first and second depth distribution information obtained by a DOE under test also falls within the distribution range of the second difference, it indicates that the DOE under test is also an abnormal DOE. In other words, based on the distribution range of the second difference, a difference range that does not belong to an abnormal DOE can be deduced. If the difference between the first depth distribution information of the DOE under test and the second distribution information of a normal DOE is within this preset range, it indicates that the DOE under test is normal; otherwise, it indicates that the DOE under test is abnormal.

[0140] Based on the above embodiments, as an optional embodiment, the preset difference range is obtained according to the following method:

[0141] Determine the second difference between any pairwise second depth distribution information and any pairwise third depth distribution information;

[0142] The upper limit of the preset difference range is determined based on the minimum value among all the second differences.

[0143] It should be understood that since there are more than one second depth distribution and one third depth distribution, there can be multiple second differences between any two second depth distributions and any two third depth distributions. Specifically, if the number of second depth distributions is 'a' and the number of third depth distributions is 'b', then at most a×b second differences can be obtained. By obtaining the minimum value among all second differences, it means that the minimum difference between the abnormal DOE and the normal DOE is obtained. Taking the minimum value among all second differences as the upper limit of the preset difference range means that if the difference between the first depth distribution of the DOE under test and the second depth distribution of the normal DOE does not exceed the upper limit of the preset difference range, then the DOE under test is not an abnormal DOE and can be identified as a normal DOE.

[0144] Based on the above embodiments, as an optional embodiment, this application embodiment also supports a step of fine-tuning a preset difference range, specifically including:

[0145] S201. Obtain at least one fourth depth map set based on natural scenes captured by normal DOE, and determine the fourth depth distribution information for each fourth depth map set.

[0146] Since the second and third depth maps are obtained based on calibrated scenes, they cannot completely and accurately reflect the distribution of depth information obtained by the DOE in natural scenes to a certain extent. Therefore, this application embodiment further obtains at least one fourth depth map based on natural scenes captured by normal DOE.

[0147] As can be seen from the above embodiments, the natural scenarios described in this application can be everyday face recognition scenarios—such as face payment scenarios, access control scenarios, terminal unlocking scenarios, etc. In these scenarios, the distance between the face and the DOE is generally between 25 cm and 50 cm. That is, each depth map in the first depth map set is acquired by the DOE within a distance of 25 cm to 50 cm from the scene object—the face, and each depth map contains information related to the distance from the surface of the scene object to the viewpoint (equivalent to the DOE). It should also be emphasized that all depth maps in the fourth depth map set obtained in this application embodiment are depth maps that have undergone information anonymization processing.

[0148] S202, Determine the third difference between at least one second depth distribution information and at least one fourth distribution information.

[0149] This application embodiment further determines the difference between the second depth distribution information and the fourth depth distribution information, which can reflect the difference in depth values ​​generated by normal DOE shooting of normal scenes and calibration scenes.

[0150] S203. Based on the distribution of all the third differences, adjust the initial upper limit of the preset difference range and determine the lower limit of the preset difference range.

[0151] Please see Figure 9 The figure exemplarily illustrates a flowchart of a fault detection method according to another embodiment of the present application, as shown in the figure, which includes:

[0152] S301. Obtain at least one second depth map set based on the normal DOE shooting calibration scene;

[0153] S302. Obtain at least one fourth depth map set based on normal DOE shooting of natural scenes;

[0154] S303. Damage the DOE to different degrees and types, and obtain at least one third depth map based on the damaged DOE to calibrate the scene;

[0155] S304. Obtain the depth distribution information of the second depth map set, the third depth map set, and the fourth depth map set respectively;

[0156] S305. Determine an initial preset difference range based on the difference between the second depth distribution information and the third depth distribution information, and adjust the initial preset difference range based on the fourth depth distribution information to obtain the adjusted preset difference range.

[0157] S306. For the DOE to be detected, obtain the first depth map set of the natural scene captured by the DOE to be detected, and determine the first depth distribution information of the first depth map set.

[0158] S307. Determine the first difference between the first depth distribution information and the second depth distribution information;

[0159] S308. Determine whether the DOE to be detected is normal based on whether the first difference is within the preset difference range. Specifically, if the first difference is within the preset difference range, the DOE to be detected is normal; if the first difference exceeds the preset difference range, the DOE to be detected is abnormal.

[0160] Based on the above embodiments, as an optional embodiment, adjusting the initial upper limit of the preset difference range and determining the lower limit of the preset difference range according to the distribution of the third difference includes:

[0161] The minimum value among all third differences is used as the lower limit of the preset difference threshold.

[0162] The smaller of the maximum value among all third differences and the initial upper limit value is used as the upper limit value of the preset difference threshold.

[0163] Please see Figure 10 The figure illustrates an embodiment of the present application that sets a preset difference threshold based on the second difference and the third difference. As shown in the figure, the coordinate axis from left to right represents the degree of difference from small to large. Coordinates 1 and 2 on the coordinate axis represent the maximum and minimum values ​​of all third differences, respectively. Coordinates 3 and 4 on the coordinate axis represent the maximum and minimum values ​​of all second differences.

[0164] The distribution of the third difference can illustrate the reasonable range of the difference between the depth information of the normal scene and the calibration scene captured by normal DOE. The minimum value among all the third differences represents the ideal difference between the depth information of the normal scene captured by normal DOE and the calibration filter. Therefore, this value is used as the lower limit of the preset difference threshold.

[0165] The initial upper limit value in this embodiment is coordinate 4. By selecting the smaller value between coordinate 1 and coordinate 4 as the upper limit value, the logic for determining whether the DOE is normal can be balanced by considering both the second and third differences. Therefore, in Figure 10 In the above, if coordinate 4 is to the right of coordinate 1, then a preset difference range is constructed based on the difference between coordinate 2 and coordinate 1; if coordinate 4 is to the left of coordinate 1, then a preset difference range is constructed based on the difference between coordinate 2 and coordinate 4.

[0166] This application embodiment also provides a method for adjusting the initial upper limit of a preset difference range and determining the lower limit of a preset difference range based on the distribution of a third difference, specifically:

[0167] S401. Based on the distribution information of the third difference, determine the expected value and variance σ of the distribution, and determine the distribution interval based on the 3σ principle;

[0168] S402. The larger of the upper limit of the second difference and the upper limit of the distribution interval shall be used as the upper limit of the preset difference range.

[0169] S403. The lower limit of the distribution interval is used as the lower limit of the preset difference range.

[0170] Specifically, when there are enough third differences, the distribution of the third differences satisfies a normal distribution. Therefore, the expected value μ and variance σ of the distribution can be determined. According to the 3σ principle, the distribution interval is (μ-3σ and μ+3σ). In this embodiment, the smaller value between the initial upper limit value and the upper limit value of the distribution interval (i.e., μ+3σ) is used as the upper limit value of the preset difference range, and the lower limit value of the distribution interval (i.e., μ-3σ) is used as the lower limit value of the preset difference range.

[0171] Please see Figure 11 The figure illustrates an exemplary flowchart of determining a preset difference range according to an embodiment of this application, as shown in the figure, including:

[0172] Obtain multiple second-depth distribution information, multiple third-depth distribution information, and multiple fourth-depth distribution information;

[0173] Determine the second difference between any pair of second depth distribution information and any pair of third depth distribution information. The line connecting any two second depth distribution information and third depth distribution information in the figure represents the calculation of a second difference.

[0174] The minimum value among all the second differences is taken as the initial upper limit value of the preset difference range;

[0175] Determine the third difference between at least one second depth distribution and at least one fourth distribution; the line connecting any two second depth distributions and fourth depth distributions in the figure represents the calculation of a third difference;

[0176] The present application then provides two methods for determining the preset difference range:

[0177] 1) Take the minimum value among all the third differences as the lower limit of the preset difference threshold; take the smaller value between the maximum value among all the third differences and the initial upper limit value as the upper limit of the preset difference threshold.

[0178] 2) Based on the distribution of the third difference, determine the expected value and variance σ of the distribution, and determine the distribution interval based on the 3σ principle; take the smaller value between the initial upper limit value and the upper limit value of the distribution interval as the upper limit value of the preset difference range; take the lower limit value of the distribution interval as the lower limit value of the preset difference range.

[0179] Based on the above embodiments, as an optional embodiment, calculating the difference between any two depth distribution information includes:

[0180] Construct a difference matrix, which includes the difference in the number of sampling points of the two depth distribution information in each depth value interval;

[0181] The difference between the two depth distribution information can be obtained by using the transpose of the difference matrix and the inverse of the covariance matrix of the difference matrix.

[0182] It should be understood that the method for calculating differences in the embodiments of this application is applicable to the calculation of differences between two depth distribution information in the above embodiments, such as the difference between the first depth distribution information and the second depth distribution information, the difference between the second depth distribution information and the third depth distribution information, the difference between the second depth distribution information and the fourth depth distribution information, and so on.

[0183] Since the depth distribution information records the number of sampling points in each depth value interval, a matrix can be obtained by calculating the difference between the number of sampling points in each depth interval of the two depth distribution information. In this embodiment, the difference matrix is ​​carried out.

[0184] For example, let the first depth distribution information be defined as (x1, x2, ..., xn), where x1 represents the number of all sampling points of the first depth map concentrated in interval 1, x2 represents the number of all sampling points of the first depth map concentrated in interval 2, ..., xn represents the number of all sampling points of the first depth map concentrated in interval n; and let the second depth distribution information be defined as (y1, y2, ..., yn), where y1 represents the number of all sampling points of the second depth map concentrated in interval 1, y2 represents the number of all sampling points of the second depth map concentrated in interval 2, ..., yn represents the number of all sampling points of the second depth map concentrated in interval n. Then the difference matrix can be represented as: (x1-y1, x2-y2, ..., xn-yn).

[0185] After obtaining the difference matrix, we can further obtain the transpose matrix (x1-y1, x2-y2, ..., xn-yn) of the difference matrix. T And the inverse matrix of the covariance matrix of the difference matrix ∑ -1 (x1-y1,x2–y2,…,xn-yn).

[0186] The difference between two depth distributions can be obtained by multiplying the transpose of the difference matrix and the inverse of the covariance matrix of the difference matrix, and then taking the square root. This difference is also known as the Mahalanobis distance.

[0187] Please see Figure 12 The figure exemplifies a flowchart of a fault detection method according to this application in a specific application scenario, as shown in the figure, including:

[0188] Select a specific model of facial recognition payment device;

[0189] Based on the normal DOE in this model of face payment device, multiple second depth maps are collected in the calibration scenario, and multiple third depth maps are collected in the natural scenario, that is, in the daily application scenario.

[0190] Obtain the second depth distribution information for each second depth map set and the third depth distribution information for each third depth map set;

[0191] Calculate the Mahalanobis distance between any second depth distribution information and any third depth distribution information to characterize the second difference between the second depth distribution information and the third depth distribution information;

[0192] The minimum value among all the second differences is taken as the initial upper limit value of the preset difference range;

[0193] Based on the abnormal DOE in this model of face payment device, multiple fourth depth maps were collected in the calibration scenario to obtain the fourth depth distribution information of each fourth depth map.

[0194] Calculate the Mahalanobis distance between each pair of at least one second depth distribution information and at least one fourth distribution information to characterize the third difference between the second depth distribution information and the fourth distribution information;

[0195] Based on the distribution of all third differences, adjust the initial upper limit of the preset difference range and determine the lower limit of the preset difference range to obtain the preset difference range;

[0196] For the face payment device of this model to be tested, acquire the first depth map set collected using the DOE to be tested in the face payment device;

[0197] Determine the first depth distribution information of the first depth atlas;

[0198] Calculate the Mahalanobis distance between each pair of the first depth distribution information and at least one second distribution information to characterize the first difference between the first depth distribution information and the second distribution information;

[0199] If the first difference is within the preset difference range, the DOE to be tested is determined to be normal. If the first difference is not within the preset difference range, the DOE to be tested is determined to be abnormal, and an alarm is issued that the face payment device to be tested is unavailable.

[0200] This application provides a fault detection device for a DOE (Design of Engines), such as... Figure 13 As shown, the device may include: a first depth distribution information acquisition module 1301, a first difference acquisition module 1302, and a judgment module 1303, specifically:

[0201] The first depth distribution information acquisition module 1301 is used to acquire a first depth map based on a natural scene captured by the DOE to be detected, and to determine the first depth distribution information of the first depth map.

[0202] The first difference acquisition module 1302 is used to determine the first difference between the first depth distribution information and at least one pre-acquired second depth distribution information;

[0203] The judgment module 1303 is used to determine that the DOE to be detected is normal if all the first differences do not exceed the preset difference range;

[0204] Each second depth distribution information is the depth distribution information of a second depth map obtained based on a normal DOE shooting calibration scene. The depth distribution information is used to represent the number of sampling points of the corresponding depth map in each depth value interval.

[0205] In one possible implementation, the judgment module is further configured to: determine that the DOE to be detected is abnormal if at least one first difference exceeds a preset difference range.

[0206] In one possible implementation, the preset difference range is obtained based on the second difference between at least one second depth distribution information and at least one pre-acquired third depth distribution information;

[0207] Each third depth distribution information is the depth distribution information of a third depth atlas obtained based on the abnormal DOE shooting calibration scene.

[0208] In one possible implementation, the fault detection device further includes a difference range acquisition module, which further includes:

[0209] The second difference acquisition submodule is used to determine the second difference between any pair of second depth distribution information and any third depth distribution information.

[0210] The range determination submodule is used to take the minimum value among all the second differences as the initial upper limit value of the preset difference range.

[0211] In one possible implementation, the difference range acquisition module also includes:

[0212] The fourth depth distribution information acquisition module is used to acquire at least one fourth depth map set based on natural scenes captured by normal DOE, and to determine the fourth depth distribution information of each fourth depth map set.

[0213] The third difference determination module is used to determine the third difference between at least one second depth distribution information and at least one fourth distribution information;

[0214] The upper and lower limit adjustment module is used to adjust the initial upper limit value of the preset difference range and determine the lower limit value of the preset difference range based on the distribution of all third differences.

[0215] In one possible implementation, the upper and lower limit adjustment module includes:

[0216] The first lower limit determination module is used to take the minimum value among all the third differences as the lower limit value of the preset difference threshold.

[0217] The first upper limit determination module is used to take the smaller of the maximum value in all third differences and the initial upper limit value as the upper limit value of the preset difference threshold.

[0218] In one possible implementation, the upper and lower limit adjustment module includes:

[0219] The distribution interval determination module is used to determine the expected value and variance σ of the distribution based on the distribution of the third difference, and to determine the distribution interval based on the 3σ principle;

[0220] The second upper limit determination module is used to take the smaller value between the initial upper limit value and the upper limit value of the distribution interval as the upper limit value of the preset difference range;

[0221] The second lower limit determination module is used to take the lower limit value of the distribution interval as the lower limit value of the preset difference range.

[0222] In one possible implementation, the fault detection device further includes a difference calculation module for calculating the difference between any two depth distribution information, the difference calculation module including:

[0223] The matrix construction unit is used to construct the difference matrix, which includes the difference in the number of sampling points of two depth distribution information in each depth value interval;

[0224] The difference calculation unit is used to obtain the difference between two depth distribution information based on the transpose of the difference matrix and the inverse of the covariance matrix of the difference matrix.

[0225] In one possible implementation, in the calibration scenario, the DOE and the calibration board are at a preset distance and a preset angle, the calibration board is a flat plate, and there are multiple preset distances.

[0226] The DOE fault detection device provided in this application specifically executes the process described in the above method embodiment. For details, please refer to the content of the above DOE fault detection method embodiment; it will not be repeated here. The DOE fault detection device provided in this application acquires a first depth map set based on a natural scene captured by the DOE to be detected, determines the first depth distribution information of the first depth map set, and determines the first difference between each pair of the first depth distribution information and at least one pre-acquired second depth distribution information. Each second depth distribution information is the depth distribution information of a second depth map set acquired based on a normal DOE capturing a calibration scene. The preset difference range can characterize the reasonable range of differences between the depth distribution information of a non-abnormal DOE and the depth distribution information of a normal DOE. Therefore, if all first differences do not exceed the preset difference range, the DOE to be detected is determined to be normal. This solution does not require additional hardware assistance for detection, has a low detection cost, and only requires a few depth maps of the DOE to be detected to accurately determine whether the DOE is faulty, resulting in higher detection efficiency and significant application value.

[0227] This application provides an electronic device comprising: a memory and a processor; at least one program stored in the memory, which, when executed by the processor, can achieve the following compared to existing technologies: by acquiring a first depth map set of a natural scene captured by a DOE to be detected, determining first depth distribution information of the first depth map set, determining a first difference between each pair of the first depth distribution information and at least one pre-acquired second depth distribution information, wherein each second depth distribution information is the depth distribution information of a second depth map set acquired based on a normal DOE capturing a calibration scene, and a preset difference range can characterize the reasonable range of difference between the depth distribution information of a non-abnormal DOE and the depth distribution information of a normal DOE. Therefore, if all first differences do not exceed the preset difference range, the DOE to be detected is determined to be normal. This solution does not require additional hardware assistance for detection during application, has a lower detection cost, and only requires a few depth maps of the DOE to be detected to accurately determine whether the DOE is faulty, resulting in higher detection efficiency and significant potential for widespread application.

[0228] In one alternative embodiment, an electronic device is provided, such as Figure 14 As shown, Figure 14 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of this electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0229] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0230] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0231] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0232] The memory 4003 stores application code that executes the scheme of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0233] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments. Compared with the prior art, by acquiring a first depth map set based on a natural scene captured by a DOE to be detected, determining the first depth distribution information of the first depth map set, and determining the first difference between each pair of the first depth distribution information and at least one pre-acquired second depth distribution information, where each second depth distribution information is the depth distribution information of a second depth map set acquired based on a normal DOE capturing a calibration scene, the preset difference range can characterize the reasonable range of difference between the depth distribution information of a non-abnormal DOE and the depth distribution information of a normal DOE. Therefore, if all first differences do not exceed the preset difference range, the DOE to be detected is determined to be normal. This solution does not require additional hardware assistance for detection during application, has a lower detection cost, and only requires a few depth maps of the DOE to be detected to accurately determine whether the DOE is faulty, resulting in higher detection efficiency and significant application value.

[0234] This application provides a computer program including computer instructions stored in a computer-readable storage medium. When a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform the content shown in the foregoing method embodiments. Compared with the prior art, by acquiring a first depth map set based on a natural scene captured by a DOE to be detected, determining the first depth distribution information of the first depth map set, and determining the first difference between each pair of the first depth distribution information and at least one pre-acquired second depth distribution information, where each second depth distribution information is the depth distribution information of a second depth map set acquired based on a normal DOE capturing a calibration scene, the preset difference range can characterize the reasonable range of difference between the depth distribution information of a non-abnormal DOE and the depth distribution information of a normal DOE. Therefore, if all first differences do not exceed the preset difference range, the DOE to be detected is determined to be normal. This solution does not require additional hardware assistance for detection during application, has a lower detection cost, and only requires a few depth maps of the DOE to be detected to accurately determine whether the DOE is faulty, resulting in higher detection efficiency and significant application value.

[0235] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0236] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A fault detection method for a diffractive optical element (DOE), characterized in that, include: Acquire a first depth map set based on the natural scene captured by the DOE to be detected, and determine the first depth distribution information of the first depth map set; Determine the first difference between the first depth distribution information and at least one pre-acquired second depth distribution information; If all of the first differences do not exceed the preset difference range, then the DOE to be detected is determined to be normal; Wherein, each of the second depth distribution information is the depth distribution information of a second depth map set obtained based on a normal DOE shooting calibration scene, and the depth distribution information is used to represent the number of sampling points of the corresponding depth map set in each depth value interval; The preset difference range is obtained based on the second difference between at least one second depth distribution information and at least one pre-acquired third depth distribution information; Each of the aforementioned third depth distribution information is the depth distribution information of a third depth atlas obtained based on an abnormal DOE shooting calibration scene.

2. The fault detection method according to claim 1, characterized in that, Also includes: If at least one of the first differences exceeds the preset difference range, then the DOE to be detected is determined to be abnormal.

3. The fault detection method according to claim 1, characterized in that, It also includes determining the preset difference range: Determine the second difference between any pair of the second depth distribution information and any pair of the third depth distribution information; The minimum value among all the second differences is taken as the initial upper limit of the preset difference range.

4. The fault detection method according to claim 3, characterized in that, The step of using the minimum value among all second differences as the initial upper limit of the preset difference range further includes: Acquire at least one fourth depth map set based on natural scenes captured by normal DOE, and determine the fourth depth distribution information for each of the fourth depth maps sets; Determine the third difference between at least one second depth distribution information and at least one fourth distribution information pairwise; Based on the distribution of all the third differences, adjust the initial upper limit of the preset difference range and determine the lower limit of the preset difference range.

5. The fault detection method according to claim 4, characterized in that, The step of adjusting the initial upper limit of the preset difference range and determining the lower limit of the preset difference range based on the distribution of the third difference includes: The minimum value among all third differences is used as the lower limit of the preset difference threshold. The smaller of the maximum value among all third differences and the initial upper limit value is used as the upper limit value of the preset difference threshold.

6. The fault detection method according to claim 4, characterized in that, The step of adjusting the initial upper limit of the preset difference range and determining the lower limit of the preset difference range based on the distribution of the third difference includes: Based on the distribution of the third difference, determine the expected value and variance σ of the distribution, and determine the distribution interval based on the 3σ principle; The smaller of the initial upper limit value and the upper limit value of the distribution interval shall be used as the upper limit value of the preset difference range; The lower limit of the distribution interval is used as the lower limit of the preset difference range.

7. The fault detection method according to any one of claims 1-6, characterized in that, Calculate the difference between any two depth distributions, including: Construct a difference matrix, which includes the difference in the number of sampling points of two depth distribution information in each depth value interval; The difference between the two depth distribution information is obtained by using the transpose of the difference matrix and the inverse of the covariance matrix of the difference matrix.

8. The fault detection method according to claim 1, characterized in that, In the calibration scenario, the DOE and the calibration board are at a preset distance and a preset angle. The calibration board is a flat plate, and there are multiple preset distances.

9. A fault detection device for a DOE (Design of Equipment), characterized in that, include: The first depth distribution information acquisition module is used to acquire a first depth map set based on the natural scene captured by the DOE to be detected, and to determine the first depth distribution information of the first depth map set. The first difference acquisition module is used to determine the first difference between the first depth distribution information and at least one pre-acquired second depth distribution information; The judgment module is used to determine that the DOE to be detected is normal if all the first differences do not exceed a preset difference range; Each of the second depth distribution information is the depth distribution information of a second depth map obtained based on a normal DOE shooting calibration scene. The depth distribution information is used to represent the number of sampling points of the corresponding depth map in each depth value interval. The preset difference range is obtained based on the second difference between at least one second depth distribution information and at least one pre-acquired third depth distribution information; Each of the aforementioned third depth distribution information is the depth distribution information of a third depth atlas obtained based on an abnormal DOE shooting calibration scene.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the fault detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to perform the steps of the fault detection method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium. When a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform the steps of implementing the fault detection method as described in any one of claims 1 to 8.

Images