Depth map detection method, device, storage medium and electronic device

Abstract

A depth map detection method, device, storage medium and electronic device, wherein a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map are obtained; a light intensity reference pixel point is determined from the reflected light intensity map according to the reflected light intensity map; and abnormal pixel points with abnormal depth values ​​are determined from the time-of-flight depth map according to the light intensity values ​​of the light intensity reference pixel points.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to a depth map detection method, device, storage medium, and electronic device. Background Art

[0002] With the rapid development of computer technology, the acquisition of depth information has not only become the key to three-dimensional vision tasks, but also plays an increasingly important role in traditional color-based vision tasks. Previously popular structured light and stereo vision either have a too small ranging range or require scene texture. Time-of-flight technology overcomes these shortcomings and has become the most promising depth acquisition method.

[0003] A time-of-flight camera transmits a modulated light signal from its transmitter, which is then reflected by objects in the scene and received by the receiver. The received light signal is then interpreted to generate a depth map of the scene. The pixel values ​​in this depth map describe the depths of different objects in the scene. However, the depth map acquisition process of a time-of-flight camera is susceptible to interference, which may result in pixels with abnormal depth values. These pixels need to be identified. Summary of the Invention

[0004] The present application provides a depth map detection method, device, storage medium and electronic device, which can screen out pixels with accurate depth information and facilitate the use of pixel depth information.

[0005] In a first aspect, the present application provides a depth map detection method, comprising:

[0006] Obtaining a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map;

[0007] determining a light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map;

[0008] Abnormal pixels with abnormal depth values ​​are determined from the time-of-flight depth map according to the light intensity values ​​of the light intensity reference pixels.

[0009] In a second aspect, the present application provides a depth map detection device, comprising:

[0010] An image acquisition module, configured to acquire a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map;

[0011] a first determining module, configured to determine a light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map;

[0012] The second determining module is configured to determine abnormal depth pixels from the time-of-flight depth map according to the light intensity values ​​of the light intensity reference pixels.

[0013] In a third aspect, the present application provides a storage medium on which a computer program is stored. When the computer program is loaded by a processor of an electronic device, the steps in any depth map detection method provided in the present application are executed.

[0014] In a fourth aspect, the present application also provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the steps of any depth map detection method provided in the present application by loading the computer program stored in the memory.

[0015] In this application, it is taken into account that there are objects of different depths in real scenes, and objects of different depths have different reflection capabilities for light signals, and reflected light signals of different light intensities will affect the depth resolution of the time-of-flight camera, thereby generating pixels with abnormal depth values. Therefore, this application uses light intensity as the basis to identify pixels with abnormal depth values. Among them, the time-of-flight depth map generated by the time-of-flight camera and the reflected light intensity corresponding to the time-of-flight depth map are first obtained, and then the light intensity reference pixel point is determined from the reflected light intensity map based on the reflected light intensity map, and the abnormal pixel point with abnormal depth value is determined from the time-of-flight depth map based on the light intensity value of the light intensity reference pixel point, thereby realizing the identification of abnormal depth value pixel points. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 Schematic diagram of the depth map detection method provided in the embodiment of the present application;

[0018] Figure 2 This is an example diagram of the signal light transmission principle in the embodiment of the present application;

[0019] Figure 3 This is an example diagram of the change in depth value before and after reflected light interference in an embodiment of the present application;

[0020] Figure 4 This is a comparative example diagram of determining a reference sub-region based on reflected light and determining a reference sub-region based on a depth value in an embodiment of the present application;

[0021] Figure 5 is a structural block diagram of a depth map detection device provided in an embodiment of the present application;

[0022] Figure 6This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] It should be noted that the terms "first," "second," and "third," etc., in this application are used to distinguish between different objects, rather than to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules is not limited to the listed steps or modules, but rather some embodiments may include steps or modules not listed, or some embodiments may include other steps or modules that are inherent to the process, method, product, or apparatus.

[0024] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0025] The present invention provides a depth map detection method, a depth map detection device, a storage medium, and an electronic device. The depth map detection method may be performed by the depth map detection device provided in the present invention, or by an electronic device incorporating the depth map detection device. The depth map detection device may be implemented in hardware or software. The electronic device may be a device such as a smartphone, tablet computer, PDA, or laptop computer equipped with a processor and capable of processing data.

[0026] Please refer to Figure 1 , Figure 1 A flow chart of the depth map detection method provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the process of the depth map detection method provided in the embodiment of the present application can be as follows:

[0027] 101. Obtain a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map.

[0028] Among them, the time-of-flight (TOF) camera is an active optical depth distance detection device, wherein the time-of-flight camera can be an indirect time-of-flight (iTOF) camera or a direct time-of-flight (dTOF) camera. For example, in an embodiment of the present application, the time-of-flight camera includes a light transmitter and a receiver. The transmitter transmits debugged infrared light, which is reflected by the target object in the scene and then received by the receiver. The modulated signal light received by the receiver is then solved by other hardware circuits and software algorithms to generate a two-dimensional depth map.

[0029] Among them, the time-of-flight depth map means that each pixel value of the image represents the distance between a certain point in the scene and the camera. The time-of-flight depth map can be used to determine the distance between the object and the camera, realizing the function of measuring the distance of the object at a long distance.

[0030] The reflected light intensity map refers to the light intensity information of each pixel in the image, which is used to describe the intensity of the received signal light and then serve as the confidence level of the target distance measurement value. That is, in the image corresponding to the scene, each pixel corresponds to a depth value and a light intensity value. The receiving end photosensitive element ( Such as image sensors ) The light intensity value received by each pixel represents the confidence of the depth value generated by the pixel solution, that is, confidence.

[0031] The reflected light intensity of a pixel affects the depth calculation for that pixel. For example, the weaker the signal light received by the receiver, the less reliable the distance and depth values ​​calculated. Similarly, the stronger the light received by the receiver, the more reliable the depth values. For example, when the reflected light intensity is within a certain range, the generated depth and distance values ​​are reliable. However, when the reflected light intensity exceeds a certain range, overexposure occurs, affecting the depth calculation. Therefore, in this embodiment of the present application, whether overexposure has occurred and whether the depth calculation is accurate can be determined based on whether the reflected light intensity is greater than a light intensity threshold.

[0032] In addition, since the reflected light intensity information of different objects and objects with different depth information in the scene image is different, there is a situation where the reflected light of close objects covers or interferes with the imaging area of ​​distant objects. For example, the reflected light of an object in the foreground area will interfere with the background area objects close to its own edge, making the reflected light intensity information of the background area inaccurate, and the calculation of the depth value of each pixel in the scene image is affected by the emitted light intensity. Therefore, when the reflected light intensity information collected in the background area is inaccurate, the measurement of the corresponding depth information is also inaccurate.

[0033] For example, Figure 2 As shown in the figure, in a real-world scenario, there are target objects A and B that are relatively close to the time-of-flight camera, as well as background C. Based on the principle of active light ranging imaging in a time-of-flight camera, the stronger the reflected light, the higher the confidence level of the calculated depth value. This means that to obtain the depth value of a distant background or other close-up, low-reflectivity target objects, stronger reflected light is required. If the reflected light intensity reaches a certain level, or if target object A is too close to the time-of-flight camera, the light reflected by object A is so strong that even the pixel areas on the time-of-flight camera that should be receiving object B and background C will receive the reflected light from object A. Therefore, for the pixel areas of object B and background C, the light reflected from object A becomes noise, affecting the depth value calculation for these pixel areas.

[0034] For example, if the time-of-flight camera is 1m, 2m, and 5m away from the target objects A, B, and C respectively, and each of the three objects occupies one-third of the field of view, then the theoretical depth information generated is as follows: Figure 3 As shown in part a, it reflects the real distance, but in reality, since the target objects A and B are close to the camera, the reflected light at the edge of target object A will interfere with target object B or target object C, and the reflected light at the edge of target object B will interfere with target object C. The actual depth information generated is usually as follows Figure 3 As shown in part b, a transition zone is formed between target objects AB and BC respectively.

[0035] Among them, the depth information of the transition zone is usually not Figure 3 The depth information of the transition zone is generally between the depth information of the two adjacent areas. For example, the depth information of the transition zone between target objects AB is between the depth information of target objects A and B, and the depth information of the transition zone between target objects BC is between the depth information of target objects B and C. Therefore, the area of ​​the transition zone cannot be directly determined based on the depth information, that is, it is difficult to directly determine the abnormal pixel points with abnormal depth information.

[0036] Among them, due to the existence of reflected light interference, the depth value of the pixel points in the transition zone is not accurately calculated, which makes it difficult to accurately describe the distance between the object and the camera and cannot be used for ranging.

[0037] Therefore, the embodiment of the present application can be used to identify pixels with abnormal depth information (for example, identifying pixels located in the transition zone) by acquiring the time-of-flight depth map and reflected light intensity map after shooting with a time-of-flight camera, and then acquire target pixels with normal depth information.

[0038] 102. Determine a light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map.

[0039] Among them, since the high reflected light intensity can easily block or interfere with the signal light of distant objects, and the stronger the reflected light, the greater the interference range on distant objects, therefore, in an embodiment of the present application, the light intensity reference pixel points can be screened by the light intensity values ​​of the pixel points in the reflected light intensity map, so as to facilitate the subsequent determination of abnormal pixel points with abnormal depth values ​​based on the light intensity values ​​of the light intensity reference pixel points.

[0040] In the embodiment of the present application, before selecting the light intensity reference pixel point according to the light intensity value, a reference sub-region may be selected according to the light intensity value, and the light intensity reference pixel point may be selected from the reference sub-region. That is, optionally, in some embodiments of the present application, the step of "determining the light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map" includes:

[0041] Determine a reference sub-region in the reflected light intensity map whose light intensity value meets a preset condition;

[0042] All pixels or part of the pixels in the reference sub-area are determined as light intensity reference pixels.

[0043] Among them, since the light intensity value will only form interference light for distant objects after reaching a certain range, the reference sub-area that can affect the light intensity value of other areas can be determined based on the light intensity value. For example, in an embodiment of the present application, the foreground area closer to the time-of-flight camera can be selected to judge and identify the impact of the reflected light of the foreground area on the background area or the interference light of other distant object pixel areas.

[0044] Among them, in the embodiment of the present application, the light intensity value of the light intensity reference pixel point is used to characterize the light intensity information of the reference sub-area. Therefore, all the pixel points in the reference sub-area can be used as light intensity reference pixel points, or some pixel points can be selected from all the pixel points, and the light intensity values ​​of some pixel points can be used to represent the overall light intensity information of the reference sub-area. That is, in the embodiment of the present application, pixel points whose light intensity values ​​can represent the overall light intensity information of the reference sub-area can be selected as light intensity reference pixel points.

[0045] For example, in an embodiment of the present application, when it is necessary to determine the interference problem of the foreground area on the background area and to determine the affected pixels, the foreground area can be determined as a reference sub-area, and the light intensity reference pixel points can be determined from the reference sub-area. The abnormal pixel points in the background area whose depth values ​​are affected can be determined based on the light intensity values ​​of the light intensity reference pixel points, thereby realizing the positioning of the transition zone between the foreground area and the background area.

[0046] In addition, since the reflected light of close objects will affect distant objects, while the reflected light of objects at the same depth distance has less impact on themselves, in the embodiment of the present application, it is necessary to accurately select reference sub-areas that can affect the light intensity values ​​of other areas.

[0047] However, since the indirect time-of-flight (iTOF) camera calculates distance values ​​through the phase difference of reflected light, a small amount of interference light can have a significant impact on the pixel depth calculation. In other words, it is difficult to accurately divide the foreground and background areas based on the depth value. The light intensity value of each pixel is the sum of the received emitted light. In other words, each pixel needs to receive a large amount of interference light before it has a significant impact on the light intensity value of the pixel. Therefore, the fluctuation range of the light intensity value of each pixel in the scene is not large. In other words, the foreground and background areas divided according to the light intensity value are closer to the actual foreground and background areas in the scene. In other words, the division of the foreground and background areas according to the light intensity value is relatively more accurate.

[0048] For example, see Figure 4 , Figure 4 This is a comparative example diagram of determining a reference sub-region based on reflected light and determining a reference sub-region based on a depth value in an embodiment of the present application, wherein: Figure 4 The left side of the middle diagram is a schematic diagram of determining the reference sub-area based on the reflected light of the pixel point in the reflected light intensity map. Figure 4 The right side of the figure is a schematic diagram of determining the reference sub-area based on the depth value of the pixel in the time-of-flight depth map. Among them, the area corresponding to the sequence number 3 in the reflected light intensity map and the time-of-flight depth map is the ideal foreground area (i.e., foreground area 3), and the area other than the foreground area 3 in the reflected light intensity map and the time-of-flight depth map is the background area 5. Among them, in the same scene, the affected areas of the reflected light intensity map and the time-of-flight depth map are the same, such as Figure 4The area of ​​the two arc-shaped areas except the foreground area 3 (hereinafter referred to as the interference area 1), and for the affected interference area 1, due to the greater influence of interference light on the depth information solution, the depth information of each pixel in the area shows obvious differences in the case of a small amount of interference light. For example, the depth values ​​of the pixels located in the background area 5 were originally relatively close, but due to the presence of interference light, the depth value solution of the interference area 1 is wrong, that is, the depth value of the pixel in the interference area 1 after solution is quite different from the depth value of the background area 5. Therefore, when dividing the area according to the depth value, it is easy to divide the interference area 1 and the background area 5 into different areas (but in fact the interference area 1 is still part of the background area 5). For example, the area 4 close to the outline of the interference area 1 is divided into the foreground area (that is, area 4 is the foreground area 4 after division according to the light intensity value). Therefore, the background area after division according to the depth value is quite different from the actual background area, and there is an inaccurate area division.

[0049] Since the fluctuation range of the light intensity value of each pixel point is small after being affected by the interference light, the light intensity value of the pixel point in the interference area is still close to the light intensity value of the background area. Therefore, when dividing the area according to the light intensity value, the interference area will still be divided into the background area. For example, see Figure 4 When determining the foreground area based on the reflected light, area 2 close to the foreground area 3 will be divided into the foreground area (that is, area 2 is the foreground area 2 divided according to the reflected light). After comparing the foreground area 2 divided according to the light intensity value with the foreground area 4 divided according to the depth value, the foreground area divided according to the light intensity value is closer to the foreground area 3 in the real scene. That is, by dividing the foreground and background areas by the light intensity value, the accuracy of the foreground and background area division can be improved, that is, the accuracy of the reference sub-area selection is improved, and the accuracy of the light intensity reference pixel acquisition is further improved. The improvement of the accuracy of the light intensity reference pixel acquisition improves the accuracy of the depth value abnormal pixel acquisition.

[0050] In an embodiment of the present application, the reflected light intensity map and the color (RGB) image corresponding to the color camera can also be combined to determine the reference sub-region. Since the color image has a higher resolution and can provide color information, the edge area between the reference sub-region and the target sub-region will be clearer and more accurate than the reflected light intensity map. Therefore, the accuracy of edge area acquisition can be improved. For example, the background information provided by the color image is richer, which can be a plane, an inclined surface, or a randomly changing scene, or a scene with background surface fitting. Therefore, when there are high requirements for the accuracy and completeness of the time-of-flight depth map, but low requirements for the frame rate and computing resources, the reflected light intensity map in this solution can be replaced with a color image or the reflected light intensity map and the color image can be used in combination. Of course, the algorithm complexity is also higher when combined with the color image. Whether to combine can be determined based on the accuracy and requirements of the time-of-flight depth map.

[0051] 103. Determine abnormal pixels with abnormal depth values ​​from the time-of-flight depth map according to the light intensity values ​​of the light intensity reference pixels.

[0052] Among them, since different reflected light intensity values ​​have different interference area sizes on the pixel area of ​​the distant object, the size of the interference area can be determined according to the reflected light intensity value, and then the abnormal pixel point with abnormal depth value can be determined.

[0053] In the embodiment of the present application, the light intensity reference pixel point is obtained by screening from the reflected light intensity map. Therefore, the light intensity value of the light intensity reference pixel point can reflect the light intensity information of the pixel point in the reflected light intensity map.

[0054] In the embodiment of the present application, there may be multiple light intensity reference pixels. Therefore, a reference light intensity value can be obtained based on the average of the light intensity values ​​of the multiple light intensity reference pixels. Since the light intensity reference pixels are selected from the reference sub-region, and the reference light intensity value is the average of the light intensity values ​​of the multiple light intensity reference pixels, the reference light intensity value can represent the light intensity of the reference sub-region. By locating abnormal pixels with abnormal depth values ​​using the reference light intensity value that can represent the light intensity of the reference sub-region, the accuracy of abnormal pixel positioning can be improved.

[0055] In an embodiment of the present application, light intensity reference pixel points can be selected from the edge of the reference sub-region, for example, one or more pixel points are selected as light intensity reference pixel points where the reference sub-region is connected to the background region. When selecting multiple light intensity reference pixel points, multiple pixel points adjacent to the background region can be selected as light intensity reference pixel points (that is, each light intensity reference pixel point is adjacent to the background region), or a normal line can be established based on the boundary where the reference sub-region is connected to the background region, and multiple light intensity reference pixel points can be selected from the normal line direction. Since the distance between the light intensity reference pixel point and the background region is relatively close, the effectiveness of the light intensity reference pixel point selection can be improved, as well as the accuracy of the subsequent abnormal pixel point positioning.

[0056] Among them, since different reflected light intensity values ​​have different effects on the interference area, for example, when the reflected light intensity value reaches a certain value range, it can affect all objects at a long distance. Therefore, in an embodiment of the present application, the range of the affected area can be judged based on whether the reflected light intensity value reaches a certain threshold. That is, optionally, in some embodiments of the present application, the step of "determining abnormal pixels with abnormal depth values ​​from the time-of-flight depth map based on the light intensity value of the light intensity reference pixel point" includes:

[0057] determining a target sub-region adjacent to the reference sub-region according to the reflected light intensity map and the reference sub-region;

[0058] When the light intensity value of the light intensity reference pixel is greater than a first preset threshold, the pixel corresponding to the target sub-region is determined as an abnormal depth pixel;

[0059] When the light intensity value of the light intensity reference pixel is greater than the second preset threshold and less than the first preset threshold, abnormal pixels with abnormal depth values ​​are determined from the target sub-region based on the light intensity value of the light intensity reference pixel.

[0060] In an embodiment of the present application, the target sub-region is an area adjacent to the reference sub-region and the distance from the time-of-flight camera is greater than the distance between the reference sub-region and the time-of-flight camera, that is, the reflected light of the reference sub-region will interfere with the target sub-region. For example, the area adjacent to the target sub-region will be affected by the reflected light of the reference sub-region, forming a transition zone on the target sub-region, that is, abnormal pixel points with abnormal depth values ​​are formed in the target sub-region.

[0061] In which, in an embodiment of the present application, the first preset threshold and the second preset threshold can be obtained according to actual needs or experimental data. For example, by continuously adjusting the intensity value of the reflected light, the width of the transition band under each intensity value is recorded. When the width of the transition band is the width of the target sub-area, the intensity value at this time can be recorded and the intensity value can be used as the first preset threshold. When the width of the transition band is less than the width of the target sub-area, the intensity value at this time can be recorded and the intensity value can be used as the second preset threshold.

[0062] Among them, in an embodiment of the present application, the first preset threshold is greater than the second preset threshold, wherein, when the light intensity value of the light intensity reference pixel point is greater than the first preset threshold, it can be understood that the reflected light of the reference area has reached a level sufficient to affect the entire target sub-area, that is, the depth information of each pixel point in the target sub-area is incorrectly solved, that is, each pixel point in the target sub-area is an abnormal pixel point with an abnormal depth value; when the light intensity value of the light intensity reference pixel point is less than the first preset threshold and greater than the second preset threshold, it can be understood that the reflected light of the reference sub-area affects some pixels in the target sub-area, rather than all pixels in the target sub-area. Therefore, for the case where the light intensity value of the light intensity reference pixel point is greater than the second preset threshold and less than the first preset threshold, the abnormal pixel point with abnormal depth value solution in the target sub-area can be determined based on the light intensity value of the light intensity reference pixel point, that is, the area in the target sub-area affected by the interfering light is determined.

[0063] In the embodiment of the present application, the range of signal light interference can be determined according to the light intensity value, and the abnormal pixel point with abnormal depth value can be determined according to the interference range. That is, optionally, in some embodiments of the present application, the step of "determining the abnormal pixel point with abnormal depth value from the target sub-area according to the light intensity value of the light intensity reference pixel point" includes:

[0064] Determine an edge region where the reference subregion and the target subregion intersect;

[0065] Determine the target width distance according to the light intensity value of the light intensity reference pixel point;

[0066] Determining an abnormal pixel area in the target sub-area according to the edge area and the target width distance;

[0067] Determine abnormal pixel points with abnormal depth values ​​according to the abnormal pixel area.

[0068] In this embodiment of the present application, the edge area refers to the starting area affected by the interfering light, that is, the edge part of the reference sub-area or the edge part of the target sub-area. Therefore, in this embodiment of the present application, for the target sub-area, the area where the reference sub-area and the target sub-area intersect can be determined as the edge area.

[0069] Among them, by determining the width distance of the interference through the light intensity value of the light intensity reference pixel point, the abnormal pixel area can be determined according to the width distance, and then the abnormal pixel point can be determined according to the abnormal pixel area.

[0070] In an embodiment of the present application, the functional relationship between the light intensity value and the width distance of the interference area can be determined in advance based on the sample time-of-flight depth map and the reflected light intensity map corresponding to the sample time-of-flight depth map, and then the width distance corresponding to the light intensity value of the light intensity reference pixel point can be determined based on the functional relationship.

[0071] In this embodiment of the present application, the pixels included in the abnormal pixel area can be determined as abnormal pixels with abnormal depth values.

[0072] In the embodiment of the present application, the width distance of the interference area may also be determined based on the light intensity value and depth value of the light intensity reference pixel point. That is, optionally, in some embodiments of the present application, the step of “determining the target width distance based on the light intensity value of the light intensity reference pixel point” includes:

[0073] The target width distance is determined according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point.

[0074] Among them, since the closer the distance between the close-range object and the time-of-flight camera is, the greater the range of influence of the reflected light on the distant object, therefore, by combining the depth value of the light intensity reference pixel point, the accuracy of obtaining the interference range width distance can be improved.

[0075] Among them, since the light intensity of the reflected light corresponding to objects with different reflectivities is different, and the interference areas formed by the reflected light with different light intensities are different, the width distance of the interference area is also related to the reflectivity of the object. Therefore, the width distance of the interference area can be determined according to the light intensity value and the reflectivity. That is, optionally, in some embodiments of the present application, the step of "determining the target width distance according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point" includes:

[0076] Determine the reflectivity of the light intensity reference pixel point according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point;

[0077] The target width distance is determined according to the light intensity value and reflectivity of the light intensity reference pixel point.

[0078] In the embodiment of the present application, the reflectivity of the pixel point can be determined according to the light intensity value and depth value of the pixel point. The relationship between the reflectivity of the pixel point and the light intensity value confidence and depth value depth of the pixel point is as follows:

[0079] reflectivity=α*confidence*depth 2 ;

[0080] Here, α is the attenuation coefficient of light received in the surrounding and central regions. α is related to factors such as the intensity of the emitted light, the field of view angle of the reflected light, and the TX light field distribution. α can be pre-set or obtained through calibration. For example, assuming a uniform light field distribution at the transmitter end of a time-of-flight camera, the α value of each pixel on the emitted light intensity map is related to its position on the map, that is, the field of view angle at which the reflected light is received by that pixel. In this case, the α value of each pixel on a two-dimensional plane can be calculated. In practice, the optical path at the center of the field of view is the shortest, so the corresponding light intensity value is also the highest, resulting in a high confidence and high accuracy depth value. Conversely, the confidence and accuracy of the depth values ​​of the surrounding pixels are relatively low. Therefore, when designing time-of-flight camera hardware, a non-uniform transmitter light source is often designed to enhance the emitted light intensity at the surrounding areas. For this reason, a calibrated α value is more practical.

[0081] Among them, the calculation of the width and distance of the interference area through the light intensity value and reflectivity of the pixel point can improve the accuracy of the width and distance calculation.

[0082] In the embodiment of the present application, the width distance corresponding to the light intensity value and reflectivity of the light intensity reference pixel point can be determined based on the functional relationship between the width distance of the interference area, the light intensity value, and the reflectivity. That is, optionally, in some embodiments of the present application, the step of "determining the target width distance based on the light intensity value and reflectivity of the light intensity reference pixel point" includes:

[0083] Obtaining a functional relationship, wherein the functional relationship is obtained by fitting the light intensity value and reflectivity of the sample reflected light intensity image and the sample width distance corresponding to the sample reflected light intensity image;

[0084] According to the functional relationship, the target width distance corresponding to the light intensity value and reflectivity of the light intensity reference pixel point is determined.

[0085] The light intensity value, reflectivity, and sample width distance of the sample reflected light intensity image can be used to fit the functional relationship between the light intensity value, reflectivity, and sample width distance. For example, the light intensity value and reflectivity can be used as independent variables, and the sample width distance can be used as the dependent variable to fit the functional relationship between the light intensity value, reflectivity, and width distance. The fitted functional relationship can be used to determine the width distance corresponding to the light intensity value and reflectivity of the light intensity reference pixel. This width distance is the width distance of the interference area between the reference sub-region and the target sub-region.

[0086] In the embodiment of the present application, the functional relationship between the light intensity value c (confidence), the reflectivity r (reflectivity) and the sample width distance dExNum can be fitted by a polynomial, and the functional relationship is specifically as follows:

[0087] dExNum=a 00 +a 10 r+a 01 c+a 11 rc+a 21 r 2 c+a 12 rc 2 +a 22 r 2 c 2 +…;

[0088] Theoretically, the number of higher-order terms is sufficient to fit any curve. The appropriate number of terms can be selected based on the accuracy and computational resource requirements of the application, or only one term can be selected. Furthermore, further simplification is possible, for example, using only light intensity as the independent variable and fitting a formula that excludes reflectivity. This can be used in scenarios where depth map quality is not critical but algorithm complexity is required.

[0089] In the embodiment of the present application, based on the abnormal pixel point with abnormal depth value obtained, the target pixel point with normal depth value can be obtained, and the image to be processed corresponding to the time-of-flight depth map is processed according to the target pixel point. That is, optionally, in some embodiments of the present application, after the step of "determining the abnormal pixel point with abnormal depth value from the time-of-flight depth map according to the intensity value of the intensity reference pixel point", the method further includes:

[0090] Determine a target pixel point with a normal depth value based on the abnormal pixel point;

[0091] Obtaining an image to be processed corresponding to the time-of-flight depth map;

[0092] The image to be processed is subjected to image processing according to the target pixel points, wherein the image processing includes at least one of semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing.

[0093] Among them, by locating the abnormal pixel points with abnormal depth information, normal pixel points (target pixel points) with normal depth information can be obtained based on the abnormal pixel points, and image processing and other operations can be performed based on the normal pixel points to realize the application of the depth information of the normal pixel points. Among them, in the embodiment of the present application, image processing can include multiple types, for example, including but not limited to semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing, etc.

[0094] In the embodiment of the present application, after determining the width distance of the interference area, the exposure parameters of the time-of-flight camera can be adjusted according to the width distance, that is, the interference of the reflected light of the close-range object on the pixel area corresponding to the distant object is reduced. That is, optionally, in some embodiments of the present application, after the step of "determining abnormal pixels with abnormal depth values ​​from the time-of-flight depth map based on the light intensity value of the light intensity reference pixel point", the method further includes:

[0095] determining a target adjustment amount according to the target width distance;

[0096] An exposure parameter of the time-of-flight camera is adjusted according to the target adjustment amount.

[0097] Among them, in an embodiment of the present application, the exposure parameters may include exposure time. By adjusting the exposure time, the number of signal light pulses can be adjusted. For example, reducing the exposure time can reduce the time for light wave emission and reception, and reducing the time for light wave emission and reception can reduce the number of signal light pulses corresponding to the reflected light. The reduction in the number of signal light pulses can reduce the interference of the reflected light of a close object on the pixel area corresponding to a distant object.

[0098] In the embodiment of the present application, the target adjustment amount refers to the numerical amplitude to be adjusted, that is, the numerical amplitude to be adjusted of the exposure parameter is determined according to the width distance affected by the interference light. By adjusting the exposure parameter, the range of the interference area is reduced, and the number of pixels affected when calculating the depth information is reduced.

[0099] Among them, in the embodiment of the present application, after determining the interference area width distance, adjustment amount and exposure parameters, the interference area width distance, adjustment amount and exposure parameters can be used as an exposure adjustment strategy to enhance the practicality of the time-of-flight camera.

[0100] In the embodiment of the present application, after locating the abnormal pixel with an abnormal depth value, the depth value of the abnormal pixel may be adjusted to improve the accuracy of the depth information of the time-of-flight depth map. That is, optionally, in some embodiments of the present application, after the step of "determining the abnormal pixel with an abnormal depth value from the time-of-flight depth map according to the light intensity value of the light intensity reference pixel", the method further includes:

[0101] A depth reference pixel corresponding to the abnormal pixel is determined from the time-of-flight depth map, and a depth value of the abnormal pixel is corrected according to the depth value of the depth reference pixel to obtain a corrected pixel.

[0102] Among them, by adjusting the depth value of the abnormal pixel point, the accuracy of the depth value of the pixel point is improved, and by selecting the depth reference pixel point through the time-of-flight depth map, the accuracy of the depth reference pixel point selection is improved, thereby improving the accuracy of the depth value correction of the abnormal pixel point.

[0103] In the embodiment of the present application, a depth reference pixel point may be selected within a preset distance range from the abnormal pixel point. That is, optionally, in some embodiments of the present application, the step of “determining a depth reference pixel point corresponding to the abnormal pixel point from the time-of-flight depth map” includes:

[0104] For the abnormal pixel, a depth reference pixel is selected from a preset distance range according to the illumination intensity of the abnormal pixel, wherein the depth reference pixel is different from the abnormal pixel.

[0105] Among them, in an embodiment of the present application, a pixel point with a light intensity value close to that of the abnormal pixel point can be selected as a depth reference pixel point. Since the close light intensity values ​​within a certain range indicate that the probability that the two points belong to the same object is high, they should have a distance value that is the same as or very close to that of the time-of-flight camera. Therefore, a pixel point with a light intensity value close to that of the abnormal pixel point can be selected as a depth reference pixel point within a certain distance range of the abnormal pixel point.

[0106] In the embodiment of the present application, the depth value of the depth reference pixel may be used as the depth value of the abnormal pixel, that is, the depth value of the abnormal pixel may be directly adjusted according to the depth value of the depth reference pixel.

[0107] In an embodiment of the present application, multiple depth reference pixel points can be selected, and the average of the depth values ​​of the multiple depth reference pixel points is used as the depth value of the abnormal pixel point. The selection of multiple depth reference pixel points improves the accuracy of depth value correction of the abnormal pixel point.

[0108] Among them, in the embodiment of the present application, the preset distance range can be adjusted or pre-established according to actual needs. For example, in the embodiment of the present application, the preset distance range can be set smaller to ensure that the distance between the depth reference pixel and the abnormal pixel is closer, thereby improving the reference value of the depth reference pixel. The preset distance range can also be set large enough, for example, the preset distance range is set to the range of the entire reflected light intensity map, and the depth reference pixel is selected from the entire reflected light intensity map.

[0109] For example, in an embodiment of the present application, a nearest neighbor backfilling method can be used, that is, backfilling starts from the edge of the transition zone adjacent to the target sub-area, and according to the light intensity value of the abnormal pixel point, a depth reference pixel point closest to the light intensity value of the abnormal pixel point is searched within a certain neighborhood range, and the depth value of the depth reference pixel point is assigned to the abnormal pixel point. The basis of this approach is that two pixels with close light intensity values ​​within a small range indicate that the probability that the two pixels belong to the same object is high, and they should have the same or very close distance values. In the case of multiple abnormal pixels, each abnormal pixel point can be backfilled in turn. For example, the backfill result of the previous pixel point can be used as the input information of the next pixel point.

[0110] Among them, in the embodiment of the present application, the nearest average backfill method can also be used, that is, the average value of the depth values ​​of all pixels in a certain neighborhood around the abnormal pixel point that are close to the abnormal pixel point's light intensity value is used as the backfill value, or the depth values ​​of these neighboring pixels whose depth values ​​are significantly different from the average depth value can be eliminated and then averaged, etc.

[0111] Among them, in an embodiment of the present application, a full-image background statistical backfilling method can also be used, that is, based on the light intensity value of a certain abnormal pixel point or the average light intensity value in a small area where a certain abnormal pixel point is located, or the average light intensity value of all abnormal pixel points, search for reference pixel points in the total reflection light intensity map that are close to the light intensity value or the average light intensity value, obtain the average depth value of these reference pixel points, assign the average depth value to the abnormal pixel point, or fit a surface function according to the position of the reference pixel point, and then estimate a depth value based on the fitted surface function and the position of the abnormal pixel point, and assign the estimated depth value to the abnormal pixel point.

[0112] In the embodiment of the present application, there may be multiple depth reference pixels. Therefore, the depth value change trend of the pixel in the reflected light intensity map may be determined based on the depth values ​​of the multiple depth reference pixels, and the depth value of the abnormal pixel may be adjusted based on the depth value change trend. That is, optionally, in some embodiments of the present application, the depth reference pixel may include a first depth reference pixel and a second depth reference pixel. The step of "determining the depth reference pixel corresponding to the abnormal pixel from the time-of-flight depth map, and correcting the depth value of the abnormal pixel according to the depth value of the depth reference pixel to obtain a corrected pixel" includes:

[0113] For the abnormal pixel, determining a first depth reference pixel and a second depth reference pixel based on a distance from the abnormal pixel, wherein a difference in depth values ​​between the first depth reference pixel and the second depth reference pixel is greater than a third preset threshold;

[0114] Establishing a line connecting the first depth reference pixel and the second depth reference pixel, and determining a depth information change trend of a plurality of pixels on the line;

[0115] Determining a target depth value according to a change trend of the depth information;

[0116] The depth value of the abnormal pixel point is adjusted to the target depth value to obtain a corrected pixel point.

[0117] The third preset threshold is set to reflect the presence of a depth difference between the first depth reference pixel and the second depth reference pixel, so as to facilitate determining the depth information change trend based on the depth difference. In an embodiment of the present application, the third preset threshold can be set according to actual needs. For example, when it is necessary to select a first depth reference pixel from a reference sub-region and a second depth reference pixel from a target sub-region, the third preset threshold can be set to the depth difference between the pixels in the reference sub-region and the pixel in the target sub-region, wherein the depth difference can be the difference between the depth values ​​of a single pixel in the two regions, or the difference after averaging the depth values ​​of multiple pixels.

[0118] Among them, by establishing a line passing through abnormal pixels, the changing trend of the depth values ​​of the pixels on the line can be determined according to the depth values ​​of each pixel on the line. Based on the changing trend, the depth values ​​of the abnormal pixels on the line with abnormal depth values ​​can be filled or backfilled according to the depth values ​​of the pixels with normal depth values ​​on the line. For example, in an embodiment of the present application, a straight line function is established based on the position information and depth information of the first depth reference pixel and the second depth reference pixel, and an estimated depth value is obtained based on the straight line function and the position information of the abnormal pixel (for the abnormal pixel on the above-mentioned straight line), and the estimated depth value is used as the depth value of the abnormal pixel.

[0119] In which, in an embodiment of the present application, the first depth reference pixel point may be located in the reference sub-area, the first depth reference pixel point may be located in the target sub-area and different from the abnormal pixel point, therefore, the depth value of the abnormal pixel point located between the third pixel point and the fourth pixel point may be determined based on the trend of the pixel depth value change between the first depth reference pixel point and the second depth reference pixel point.

[0120] For example, in an embodiment of the present application, a linear backfilling method can be used, that is, the depth values ​​of the two pixels closest to the abnormal pixel (one of which is located in the reference sub-region and the other is located in the target sub-region and is not an abnormal pixel) are taken out, and all abnormal pixels crossed by the line connecting these two pixels are assigned depth values ​​that gradually approach the background depth value to the foreground depth value in sequence according to the linear gradient method. Of course, the selection of pixels in the reference sub-region and the target sub-region can also be limited to the same row or column as the abnormal pixel (wherein, the same row and column refer to being on the same horizontal axis or the same vertical axis as the abnormal pixel in the coordinates), rather than directly selecting the nearest pixel, etc.

[0121] In the embodiment of the present application, before adjusting the depth value of the abnormal pixel, the abnormal pixel can be marked. The marking of the abnormal pixel facilitates the correction of the depth value of the abnormal pixel and improves the correction efficiency.

[0122] In the embodiment of the present application, for the pixel points after the depth value is corrected, the pixel points after the depth value is corrected can be selected for image processing or ranging application. That is, optionally, in some embodiments of the present application, after the step of "obtaining the corrected pixel points", the method further includes:

[0123] Obtaining a corrected time-of-flight depth map according to the corrected pixel points;

[0124] Obtaining an image to be processed corresponding to the corrected time-of-flight depth map;

[0125] Performing image processing on the image to be processed according to the corrected time-of-flight depth map, wherein the image processing includes at least one of semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing.

[0126] Among them, after correcting the depth values ​​of pixels with abnormal depth values, a corrected time-of-flight depth map can be obtained. Image processing and other operations can be performed based on the corrected time-of-flight depth map to realize the application of pixel depth information. Among them, in an embodiment of the present application, image processing may include multiple types, for example, including but not limited to semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing, etc.

[0127] Among them, in this application, it is taken into account that there are objects of different depths in the real scene, and objects at different depths have different reflection capabilities for light signals, and reflected light signals of different light intensities will affect the depth resolution of the time-of-flight camera, thereby generating pixels with abnormal depth values. Therefore, this application uses light intensity as the basis to identify pixels with abnormal depth values. Among them, first, the time-of-flight depth map generated by the time-of-flight camera and the reflected light intensity corresponding to the time-of-flight depth map are obtained, and then the light intensity reference pixel point is determined from the reflected light intensity map based on the reflected light intensity map, and based on the light intensity value of the light intensity reference pixel point, the abnormal pixel point with abnormal depth value is determined from the time-of-flight depth map, thereby realizing the identification of abnormal depth value pixel points.

[0128] To facilitate better implementation of the depth map detection method of this application, this application also provides a depth map detection device based on the above-mentioned depth map detection method. The meanings of the terms in the depth map detection device are the same as those in the above-mentioned depth map detection method. For specific implementation details, please refer to the description in the method embodiment.

[0129] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of the depth map detection device provided by the present application, wherein the depth map detection device may include:

[0130] An image acquisition module 201 is configured to acquire a time-of-flight depth map and a reflected light intensity map corresponding to the time-of-flight depth map from a time-of-flight camera;

[0131] A first determining module 202 is configured to determine a light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map;

[0132] The second determining module 203 is configured to determine abnormal pixels having abnormal depth values ​​from the time-of-flight depth map according to the light intensity values ​​of the light intensity reference pixels.

[0133] Optionally, in one embodiment, the first determining module 202 is specifically configured to:

[0134] Determine a reference sub-region in the reflected light intensity map whose light intensity value meets a preset condition;

[0135] All pixels or part of the pixels in the reference sub-area are determined as light intensity reference pixels.

[0136] Optionally, in one embodiment, the second determining module 203 is specifically configured to:

[0137] determining a target sub-region adjacent to the reference sub-region according to the reflected light intensity map and the reference sub-region;

[0138] When the light intensity value of the light intensity reference pixel is greater than a first preset threshold, the pixel corresponding to the target sub-region is determined as an abnormal depth pixel;

[0139] When the light intensity value of the light intensity reference pixel is greater than the second preset threshold and less than the first preset threshold, abnormal pixels with abnormal depth values ​​are determined from the target sub-region based on the light intensity value of the light intensity reference pixel.

[0140] Optionally, in one embodiment, the second determining module 203 is specifically configured to:

[0141] determining a target sub-region adjacent to the reference sub-region according to the reflected light intensity map and the reference sub-region;

[0142] When the light intensity value of the light intensity reference pixel is greater than a first preset threshold, the pixel corresponding to the target sub-region is determined as an abnormal depth pixel;

[0143] When the light intensity value of the light intensity reference pixel is greater than the second preset threshold and less than the first preset threshold, abnormal pixels with abnormal depth values ​​are determined from the target sub-region based on the light intensity value of the light intensity reference pixel.

[0144] Optionally, in one embodiment, the second determining module 203 is specifically configured to:

[0145] Determine an edge region where the reference subregion and the target subregion intersect;

[0146] Determine the target width distance according to the light intensity value of the light intensity reference pixel point;

[0147] Determining an abnormal pixel area in the target sub-area according to the edge area and the target width distance;

[0148] Determine abnormal pixel points with abnormal depth values ​​according to the abnormal pixel area.

[0149] Optionally, in one embodiment, the second determining module 203 is specifically configured to:

[0150] The target width distance is determined according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point.

[0151] Optionally, in one embodiment, the second determining module 203 is specifically configured to:

[0152] Determine the reflectivity of the light intensity reference pixel point according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point;

[0153] The target width distance is determined according to the light intensity value and reflectivity of the light intensity reference pixel point.

[0154] Optionally, in one embodiment, the second determining module 203 is specifically configured to:

[0155] Obtaining a functional relationship, wherein the functional relationship is obtained by fitting the light intensity value and reflectivity of the sample reflected light intensity image and the sample width distance corresponding to the sample reflected light intensity image;

[0156] According to the functional relationship, the target width distance corresponding to the light intensity value and reflectivity of the light intensity reference pixel point is determined.

[0157] Optionally, in one embodiment, the device further includes a first application module, and the first application module is specifically configured to:

[0158] Determine a target pixel point with a normal depth value based on the abnormal pixel point;

[0159] Obtaining an image to be processed corresponding to the time-of-flight depth map;

[0160] The image to be processed is subjected to image processing according to the target pixel points, wherein the image processing includes at least one of semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing.

[0161] Optionally, in one embodiment, the device further includes an adjustment module, and the adjustment module is specifically configured to:

[0162] determining a target adjustment amount according to the target width distance;

[0163] An exposure parameter of the time-of-flight camera is adjusted according to the target adjustment amount.

[0164] Optionally, in one embodiment, the device further includes a backfill module, which is specifically configured to:

[0165] A backfilling unit is used to determine a depth reference pixel point corresponding to the abnormal pixel point from the time-of-flight depth map, and correct the depth value of the abnormal pixel point according to the depth value of the depth reference pixel point to obtain a corrected pixel point.

[0166] Optionally, in one embodiment, the backfill module is specifically configured to:

[0167] For the abnormal pixel, a depth reference pixel is selected from a preset distance range according to the illumination intensity of the abnormal pixel, wherein the depth reference pixel is different from the abnormal pixel.

[0168] Optionally, in one embodiment, the backfill module is specifically configured to:

[0169] For the abnormal pixel, determining a first depth reference pixel and a second depth reference pixel according to a distance from the abnormal pixel, wherein the first depth reference pixel and the second depth reference pixel are different from the abnormal pixel;

[0170] Establishing a line connecting the first depth reference pixel and the second depth reference pixel, and determining a depth information change trend of a plurality of pixels on the line;

[0171] The depth value of the abnormal pixel is adjusted according to the change trend of the depth information.

[0172] Optionally, in one embodiment, the device further includes a second application module, and the second application module is specifically configured to:

[0173] Obtaining a corrected time-of-flight depth map according to the corrected pixel points;

[0174] Obtaining an image to be processed corresponding to the corrected time-of-flight depth map;

[0175] Performing image processing on the image to be processed according to the corrected time-of-flight depth map, wherein the image processing includes at least one of semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing.

[0176] It should be noted that the depth map detection device provided in the embodiment of the present application and the depth map detection method in the above embodiment have the same concept, and the specific implementation process thereof is detailed in the above related embodiments and will not be repeated here.

[0177] The present invention provides a storage medium having a computer program stored thereon. When the computer program stored thereon is executed on a processor of an electronic device provided in the present invention, the processor of the electronic device performs the steps of any of the above-mentioned depth map detection methods applicable to electronic devices. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0178] This application also provides an electronic device, please refer to Figure 6 , the electronic device 300 includes a processor 310 and a memory 320.

[0179] The processor 310 in the embodiment of the present application may be a general-purpose processor, such as an ARM architecture processor.

[0180] The memory 320 stores a computer program, which may be a high-speed random access memory or a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320. The processor 310 executes the computer program in the memory 320 to perform any of the above depth map detection methods, such as:

[0181] Obtaining a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map;

[0182] determining a light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map;

[0183] Abnormal pixels with abnormal depth values ​​are determined from the time-of-flight depth map according to the light intensity values ​​of the light intensity reference pixels.

[0184] The above is a detailed introduction to the depth map detection method, device, storage medium and electronic device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A depth map detection method, characterized in that: include: Obtaining a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map; Determining light intensity reference pixels from the reflected light intensity map according to the reflected light intensity map includes: determining a reference sub-region in the reflected light intensity map whose light intensity value meets a preset condition; and determining all or part of the pixels in the reference sub-region as light intensity reference pixels; According to the light intensity value of the light intensity reference pixel point, abnormal pixel points with abnormal depth values ​​are determined from the flight time depth map, including: determining a target sub-region adjacent to the reference sub-region according to the reflected light intensity map and the reference sub-region; when the light intensity value of the light intensity reference pixel point is greater than a first preset threshold, determining the pixel point corresponding to the target sub-region as an abnormal depth pixel point; when the light intensity value of the light intensity reference pixel point is greater than a second preset threshold and less than the first preset threshold, determining abnormal pixel points with abnormal depth values ​​from the target sub-region according to the light intensity value of the light intensity reference pixel point.

2. The depth map detection method according to claim 1, wherein: The determining, from the target sub-region, abnormal pixels having abnormal depth values ​​according to the light intensity values ​​of the light intensity reference pixels, includes: Determine an edge region where the reference subregion and the target subregion intersect; Determine the target width distance according to the light intensity value of the light intensity reference pixel point; Determining an abnormal pixel area in the target sub-area according to the edge area and the target width distance; Determine abnormal pixel points with abnormal depth values ​​according to the abnormal pixel area.

3. The depth map detection method according to claim 2, wherein: The determining of the target width distance according to the light intensity value of the light intensity reference pixel point includes: The target width distance is determined according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point.

4. The depth map detection method according to claim 3, wherein: The determining of the target width distance according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point includes: Determine the reflectivity of the light intensity reference pixel point according to the light intensity value of the light intensity reference pixel point and the depth value of the light intensity reference pixel point; The target width distance is determined according to the light intensity value and reflectivity of the light intensity reference pixel point.

5. The depth map detection method according to claim 4, characterized in that: The determining of the target width distance according to the light intensity value and reflectivity of the light intensity reference pixel point includes: Obtaining a functional relationship, wherein the functional relationship is obtained by fitting the light intensity value and reflectivity of the sample reflected light intensity image and the sample width distance corresponding to the sample reflected light intensity image; According to the functional relationship, the target width distance corresponding to the light intensity value and reflectivity of the light intensity reference pixel point is determined.

6. The depth map detection method according to any one of claims 1 to 5, characterized in that: After determining abnormal pixels with abnormal depth values ​​from the time-of-flight depth map based on the light intensity values ​​of the light intensity reference pixels, the method further includes: Determine a target pixel point with a normal depth value based on the abnormal pixel point; Obtaining an image to be processed corresponding to the time-of-flight depth map; The image to be processed is subjected to image processing according to the target pixel points, wherein the image processing includes at least one of semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing.

7. The depth map detection method according to any one of claims 2 to 5, characterized in that: After determining abnormal pixels with abnormal depth values ​​from the time-of-flight depth map based on the light intensity values ​​of the light intensity reference pixels, the method further includes: determining a target adjustment amount according to the target width distance; An exposure parameter of the time-of-flight camera is adjusted according to the target adjustment amount.

8. The depth map detection method according to any one of claims 1 to 5, characterized in that: After determining abnormal pixels with abnormal depth values ​​from the time-of-flight depth map based on the light intensity values ​​of the light intensity reference pixels, the method further includes: A depth reference pixel corresponding to the abnormal pixel is determined from the time-of-flight depth map, and a depth value of the abnormal pixel is corrected according to the depth value of the depth reference pixel to obtain a corrected pixel.

9. The depth map detection method according to claim 8, wherein: The determining of a depth reference pixel corresponding to the abnormal pixel from the time-of-flight depth map includes: For the abnormal pixel, a depth reference pixel is selected from a preset distance range according to the illumination intensity of the abnormal pixel, wherein the depth reference pixel is different from the abnormal pixel.

10. The depth map detection method according to claim 8, characterized in that: The depth reference pixel includes a first depth reference pixel and a second depth reference pixel, and determining the depth reference pixel corresponding to the abnormal pixel from the time-of-flight depth map, and correcting the depth value of the abnormal pixel according to the depth value of the depth reference pixel to obtain a corrected pixel, including: For the abnormal pixel, determining a first depth reference pixel and a second depth reference pixel based on a distance from the abnormal pixel, wherein a depth value difference between the first depth reference pixel and the second depth reference pixel is greater than a third preset threshold; Establishing a line connecting the first depth reference pixel and the second depth reference pixel, and determining a depth information change trend of a plurality of pixels on the line; Determining a target depth value according to a change trend of the depth information; The depth value of the abnormal pixel point is adjusted to the target depth value to obtain a corrected pixel point.

11. The depth map detection method according to claim 8, wherein: After obtaining the corrected pixel point, the method further includes: Obtaining a corrected time-of-flight depth map according to the corrected pixel points; Obtaining an image to be processed corresponding to the corrected time-of-flight depth map; Performing image processing on the image to be processed according to the corrected time-of-flight depth map, wherein the image processing includes at least one of semantic segmentation processing, gesture recognition processing, gesture tracking processing, three-dimensional reconstruction processing, light scanning processing or defect recognition processing.

12. A depth map detection device, characterized in that: include: An image acquisition module, configured to acquire a time-of-flight depth map of a time-of-flight camera and a reflected light intensity map corresponding to the time-of-flight depth map; A first determining module is configured to determine a light intensity reference pixel point from the reflected light intensity map according to the reflected light intensity map, including: determining a reference sub-region in the reflected light intensity map whose light intensity value meets a preset condition; and determining all or part of the pixel points in the reference sub-region as the light intensity reference pixel points; The second determination module is used to determine abnormal pixels with abnormal depth values ​​from the time-of-flight depth map based on the light intensity value of the light intensity reference pixel, including: determining a target sub-region adjacent to the reference sub-region based on the reflected light intensity map and the reference sub-region; when the light intensity value of the light intensity reference pixel is greater than a first preset threshold, determining the pixel corresponding to the target sub-region as an abnormal depth pixel; when the light intensity value of the light intensity reference pixel is greater than a second preset threshold and less than the first preset threshold, determining abnormal pixels with abnormal depth values ​​from the target sub-region based on the light intensity value of the light intensity reference pixel.

13. A storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded by a processor of an electronic device, the steps of the depth map detection method according to any one of claims 1 to 11 are executed.

14. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, wherein: The processor executes the steps in the depth map detection method according to any one of claims 1 to 11 by loading the computer program.

Images