Infrared speckle depth camera shielding detection method, system and device and storage medium

Abstract

The invention discloses an infrared speckle depth camera occlusion detection method, system and device, and a storage medium. The method comprises the following steps: S1, obtaining a first infrared speckle image and a first dense depth map at a first moment; s2, extracting a first speckle center on the first infrared speckle image, calculating a first difference value between the first speckle center and surrounding pixels, and marking a first alternative area; s3, acquiring a second infrared speckle image and a second dense depth map at a second moment; s4, subtracting the first dense depth map from the second dense depth map to obtain a third dense depth map, and if the infrared speckle depth camera is in a motion state, executing the step S5; s5, extracting a second speckle center on the second infrared speckle image, calculating a second difference value between the second speckle center and surrounding pixels, and marking a second alternative area; and S6, marking the shielding area.

Description

Background Art

[0002] In the field of intelligent robots, as a key vision sensor, the infrared speckle depth camera is widely used in the navigation and obstacle avoidance systems of various robots because it can accurately obtain environmental depth information. It consists of an infrared emission and reception module working together, constructing a depth image by emitting and receiving infrared signals, and providing a basis for the robot's action decision-making.

[0003] However, due to the complex working environment, the lenses of the camera's emission and reception modules are easily blocked by dirt such as fingerprints, dust, and water droplets. Once this happens, the depth imaging quality is greatly affected, resulting in deviations or errors in the depth information. And the robot's navigation and obstacle avoidance highly rely on these data, and the wrong information will mislead its actions, possibly leading to collision accidents, seriously restricting the reliable operation of the robot.

[0004] In the prior art, detection is mainly carried out through the occlusion area, etc., and small occlusions cannot be effectively detected. Especially for the infrared speckle depth camera, the speckle size is very small, and even a small-sized occlusion will have a great impact on the depth value.

[0005] The disclosure of the above background art content is only used to assist in understanding the inventive concept and technical solution of the present invention, and it does not necessarily belong to the prior art of this patent application. Without clear evidence indicating that the above content was publicly available on the filing date of this patent application, the above background art should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0006] For this reason, the present invention proposes an occlusion detection method for an infrared speckle depth camera, which can effectively identify the dirt situation, accurately eliminate the wrong depth data caused by dirt, provide more reliable and accurate information for the robot's navigation and obstacle avoidance, and help it operate safely and efficiently in a complex environment.

[0007] In a first aspect, the present invention provides an occlusion detection method for an infrared speckle depth camera, characterized by including:

[0008] Step S1: Obtain a first infrared speckle image and a first dense depth map at a first moment; wherein, the first dense depth map and the first infrared speckle image are pixel-aligned;

[0009] Step S2: Extract a first speckle center on the first infrared speckle image, and calculate a first difference between the first speckle center and surrounding pixels; when the first difference is less than a first depth threshold, mark the speckle area where the first speckle center is located as a first alternative area;

[0010] Step S3: Obtain the second infrared speckle image and the second dense depth map at the second moment; wherein, the second dense depth map and the second infrared speckle image are pixel-aligned;

[0011] Step S4: Subtract the first dense depth map from the second dense depth map to obtain a third dense depth map. If the number of pixel points with depth values exceeding the second depth threshold in the third dense depth map exceeds the first quantity threshold, it is considered that the infrared speckle depth camera is in a moving state, and step S5 is executed;

[0012] Step S6: Extract the second speckle center on the second infrared speckle image and calculate the second difference between the second speckle center and the surrounding pixels; when the second difference is less than the first depth threshold, mark the speckle region where the second speckle center is located as the second alternative region;

[0013] Step S7: If the number of overlapping pixel points in the first alternative region and the second alternative region exceeds the second quantity threshold, mark the first alternative region and the second alternative region as occlusion regions.

[0014] Optionally, in the above-mentioned infrared speckle depth camera occlusion detection method, it further includes:

[0015] Step S7: Obtain the first sparse depth map of the first infrared speckle map and the second sparse depth map of the second infrared speckle map; wherein, the first sparse depth map and the first infrared speckle image are pixel-aligned, and the second sparse depth map and the second infrared speckle image are pixel-aligned;

[0016] Step S8: Remove the depth values of the occlusion regions in the first dense depth map and fill the corresponding depth values in the first sparse depth map into the first dense depth map; remove the depth values of the occlusion regions in the second dense depth map and fill the corresponding depth values in the second sparse depth map into the second dense depth map.

[0017] Optionally, in the above-mentioned infrared speckle depth camera occlusion detection method, the time interval between the first moment and the second moment is inversely proportional to the preset maximum movement speed of the infrared speckle depth camera.

[0018] Optionally, in the above-mentioned infrared speckle depth camera occlusion detection method, step S2 includes:

[0019] Step S21: Extract the first speckle center on the first infrared speckle image;

[0020] Step S22: Taking each of the first speckle centers as the center and a circular area with a radius of r as the neighborhood, calculate the difference in gray values between the first speckle center and other pixels within the neighborhood to obtain a first difference value.

[0021] Step S23: When the first difference value is less than a first depth threshold, mark the speckle area where the first speckle center is located as a first alternative area; wherein, the first depth threshold is dynamically adjusted according to the average depth information and speckle density of the first infrared speckle image.

[0022] Optionally, in the infrared speckle depth camera occlusion detection method, step S4 includes:

[0023] Step S41: Subtract the first dense depth map and the second dense depth map pixel by pixel to obtain a third dense depth map, and filter to eliminate jump noise.

[0024] Step S42: Count the number of pixel points in the third dense depth map whose depth values exceed a second depth threshold, that is, the exceeded number.

[0025] Step S43: If the exceeded number is greater than a first number threshold, it is considered that the infrared speckle depth camera is in a moving state, and execute step S5; wherein, the first number threshold is adjusted according to the size of the first infrared speckle map and the speckle spacing.

[0026] Optionally, in the infrared speckle depth camera occlusion detection method, in step S4, motion analysis is performed on the second infrared speckle images at multiple different second moments as the motion states of the multiple second infrared speckle maps.

[0027] Optionally, in the infrared speckle depth camera occlusion detection method, step S6 includes:

[0028] Step S61: Calculate the number of overlapping pixel points in the first alternative area and the second alternative area based on a fast overlapping calculation method based on an image mask.

[0029] Step S62: If the number of overlapping pixel points exceeds a second number threshold, mark the first alternative area and the second alternative area as occlusion areas; the second number threshold is adjusted according to the occlusion probability and the average size of the speckle area.

[0030] Step S63: Eliminate the non-corresponding closed areas in the first alternative area and the second alternative area to obtain the final occlusion area.

[0031] Second aspect, the present invention provides an infrared speckle depth camera occlusion detection system for implementing the infrared speckle depth camera occlusion detection method described in any one of the foregoing, characterized in that it includes:

[0032] A first acquisition module for acquiring a first infrared speckle image and a first dense depth map at a first moment; wherein, the first dense depth map and the first infrared speckle image are pixel-aligned;

[0033] A first difference module for extracting a first speckle center on the first infrared speckle image and calculating a first difference between the first speckle center and surrounding pixels; when the first difference is less than a first depth threshold, marking the speckle region where the first speckle center is located as a first alternative region;

[0034] A second acquisition module for acquiring a second infrared speckle image and a second dense depth map at a second moment; wherein, the second dense depth map and the second infrared speckle image are pixel-aligned;

[0035] A second difference module for subtracting the first dense depth map from the second dense depth map to obtain a third dense depth map. If the number of pixel points with depth values exceeding a second depth threshold in the third dense depth map exceeds a first quantity threshold, it is considered that the infrared speckle depth camera is in a moving state, and the calculation module is executed;

[0036] A calculation module for extracting a second speckle center on the second infrared speckle image and calculating a second difference between the second speckle center and surrounding pixels; when the second difference is less than the first depth threshold, marking the speckle region where the second speckle center is located as a second alternative region;

[0037] A marking module for marking the first alternative region and the second alternative region as occlusion regions if the number of overlapping pixel points in the first alternative region and the second alternative region exceeds a second quantity threshold.

[0038] Third aspect, the present invention provides an infrared speckle depth camera occlusion detection device, characterized in that it includes:

[0039] A processor;

[0040] A memory in which executable instructions of the processor are stored;

[0041] Wherein, the processor is configured to execute the steps of the infrared speckle depth camera occlusion detection method described in any one of the foregoing by executing the executable instructions.

[0042] Fourthly, the present invention provides a computer-readable storage medium for storing a program, characterized in that when the program is executed, the steps of the infrared speckle depth camera occlusion detection method described in any one of the foregoing are implemented.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention respectively obtains an infrared speckle image and a dense depth map at different times, and calculates the difference between the speckle center and the surrounding pixels. When the difference is less than the first depth threshold, the alternative area is marked. This kind of detailed analysis at the pixel level can accurately locate the areas where occlusion may exist, greatly improving the accuracy of occlusion detection.

[0045] The present invention subtracts the first dense depth map from the second dense depth map to obtain the third dense depth map, and judges whether the camera is in a moving state according to the number of pixel points whose depth values exceed the second depth threshold, fully considering the possible movement of the camera in practical applications, avoiding false judgments caused by camera movement, and making the occlusion detection result more reliable.

[0046] The present invention successively obtains the image and depth map information at the first moment and the second moment, not only analyzes the speckle center and area on a single-frame image, but also finally determines the occlusion area by comparing the number of overlapping pixel points in the alternative areas of the two frames, enhancing the stability and accuracy of the detection method, and reducing the influence of noise or interference that may exist in a single frame.

[0047] The present invention can complete the process from image acquisition to occlusion area judgment in a short time, meet the real-time requirements for infrared speckle depth camera occlusion detection, and is applicable to various application scenarios that require real-time feedback. Description of the Drawings

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects, and advantages of the present invention will become more obvious:

[0049] Figure 1 It is a flowchart of the steps of an infrared speckle depth camera occlusion detection method in an embodiment of the present invention;

[0050] Figure 2 It is a flowchart of the steps of another infrared speckle depth camera occlusion detection method in an embodiment of the present invention;

[0051] Figure 3 It is a flowchart of steps for marking a first alternative area in an embodiment of the present invention;

[0052] Figure 4 It is a flowchart of steps for judging a motion state in an embodiment of the present invention;

[0053] Figure 5 It is a flowchart of steps for marking an occlusion area in an embodiment of the present invention;

[0054] Figure 6 It is a schematic structural diagram of an infrared speckle depth camera occlusion detection system in an embodiment of the present invention;

[0055] Figure 7 It is a schematic structural diagram of an infrared speckle depth camera occlusion detection device in an embodiment of the present invention; and

[0056] Figure 8 It is a schematic structural diagram of a computer-readable storage medium in an embodiment of the present invention. Detailed implementation manners

[0057] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several modifications and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0058] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here, for example, can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0059] An infrared speckle depth camera occlusion detection method provided by an embodiment of the present invention aims to solve the problems existing in the prior art.

[0060] The following will specifically describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above technical problems. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below with reference to the accompanying drawings.

[0061] Based on pixel-level analysis, the present invention obtains infrared speckle images and dense depth maps at different times respectively. By calculating the difference between the speckle center and surrounding pixels, it accurately locates possible occlusion areas to achieve high-precision detection. At the same time, by comparing the dense depth maps at different times, it judges the camera motion state to avoid misjudgment caused by camera movement, making the detection result more reliable. By comprehensively using multi-frame information, it not only analyzes single-frame images, but also determines the occlusion area by comparing the number of overlapping pixel points in the alternative areas of two frames, enhancing the stability and accuracy of the detection and reducing noise interference. And it can complete the detection in a short time, with good real-time performance, and is applicable to various application scenarios that require real-time feedback.

[0062] Figure 1 It is a step flowchart of an occlusion detection method for an infrared speckle depth camera in an embodiment of the present invention. As Figure 1 shown, the steps of an occlusion detection method for an infrared speckle depth camera in an embodiment of the present invention include:

[0063] Step S1: Obtain a first infrared speckle image and a first dense depth map at a first time.

[0064] In this step, the first dense depth map and the first infrared speckle image are pixel-aligned. Pixel alignment means that each pixel point in the depth map corresponds one-to-one with the corresponding pixel point in the infrared speckle image in terms of spatial position.

[0065] The first infrared speckle image is an image formed by an infrared speckle projector projecting a speckle pattern with a specific pattern onto a scene and then an infrared camera capturing the reflected speckle pattern. The speckle pattern will be deformed due to factors such as the shape and distance of objects in the scene, and this deformation information can be used for subsequent depth calculation and analysis.

[0066] The first dense depth map records the distance information of each pixel point in the scene to the camera. It is pixel-aligned with the first infrared speckle image, which means that the pixels at the same position in the two images correspond to the same position in the scene. This alignment relationship is the basis for precise analysis in subsequent steps. The first dense depth map can be calculated from the first infrared speckle image or from other images aligned with the first infrared speckle image. For example, after the infrared speckle depth camera obtains the first infrared speckle image, it calculates the depth information based on the first infrared speckle image to obtain the first dense depth map. At this time, the first infrared speckle image and the first dense depth map are aligned.

[0067] Step S2: Extract the first speckle center on the first infrared speckle image, and calculate the first difference between the first speckle center and the surrounding pixels; when the first difference is less than the first depth threshold, mark the speckle area where the first speckle center is located as the first alternative area.

[0068] In this step, find the area where occlusion may exist from the first infrared speckle image and mark it as the first alternative area. The first speckle center is a representative position in the speckle pattern, and usually, the center position of each speckle can be determined by image processing algorithms (such as centroid calculation, peak detection, etc.). For each extracted first speckle center, calculate the difference (the first difference) between its gray value or other feature values and the surrounding pixels. The surrounding pixels can be the neighborhood pixels of the central pixel, such as 3x3, 5x5, etc. Set a first depth threshold. If the calculated first difference is less than this threshold, it means that the gray change in the area where the speckle center is located is small, and there may be situations such as occlusion. Mark the speckle area where the first speckle center is located as the first alternative area.

[0069] Step S3: Obtain the second infrared speckle image and the second dense depth map at the second moment.

[0070] In this step, the second dense depth map and the second infrared speckle image are pixel-aligned. Similar to step S1, in this step, the second infrared speckle image at the second moment is obtained through the infrared speckle projector and the infrared camera, and at the same time, the second dense depth map that is pixel-aligned with it is obtained for comparative analysis with the data at the first moment.

[0071] Step S4: Subtract the first dense depth map from the second dense depth map to obtain the third dense depth map. If the number of pixel points in the third dense depth map whose depth values exceed the second depth threshold exceeds the first quantity threshold, it is considered that the infrared speckle depth camera is in a moving state, and step S5 is executed.

[0072] In this step, according to the difference between the dense depth maps at the first moment and the second moment, it is judged whether the infrared speckle depth camera is in a moving state. Subtract the depth values of the corresponding pixels in the first dense depth map and the second dense depth map to obtain the third dense depth map. The third dense depth map reflects the change in the scene depth between the two moments. Set a second depth threshold and a first quantity threshold. Count the number of pixel points in the third dense depth map whose depth values exceed the second depth threshold. If it exceeds the first quantity threshold, it means that the depth of most areas in the scene has changed greatly, which is very likely caused by the movement of the camera. At this time, it is considered that the infrared speckle depth camera is in a moving state, and step S5 is executed.

[0073] Step S5: Extract the second speckle center from the second infrared speckle image, and calculate the second difference between the second speckle center and the surrounding pixels; when the second difference is less than the first depth threshold, mark the speckle area where the second speckle center is located as the second alternative area.

[0074] In this step, find the area where occlusion may exist from the second infrared speckle image, mark it as the second alternative area, and compare it with the first alternative area. Similar to step S2, extract the second speckle center from the second infrared speckle image, and calculate the difference between the second speckle center and the surrounding pixels (the second difference). When the second difference is less than the first depth threshold, mark the speckle area where the second speckle center is located as the second alternative area.

[0075] Step S6: If the number of overlapping pixel points in the first alternative area and the second alternative area exceeds the second quantity threshold, mark the first alternative area and the second alternative area as occlusion areas.

[0076] In this step, determine the occlusion area in the scene according to the overlapping situation of the first alternative area and the second alternative area. Set a second quantity threshold, and count the number of overlapping pixel points in the first alternative area and the second alternative area. If the number of overlapping pixel points exceeds the second quantity threshold, it means that these two areas both show the characteristics of possible occlusion at two moments, and mark the first alternative area and the second alternative area as occlusion areas.

[0077] Figure 2 This is the step flow chart of another infrared speckle depth camera occlusion detection method in the embodiments of the present invention. As Figure 2 shown, compared with the previous embodiments, the steps of another infrared speckle depth camera occlusion detection method in the embodiments of the present invention further include:

[0078] Step S7: Obtain the first sparse depth map of the first infrared speckle map and the second sparse depth map of the second infrared speckle map.

[0079] In this step, the first sparse depth map is pixel-aligned with the first infrared speckle image, and the second sparse depth map is pixel-aligned with the second infrared speckle image. Different from the dense depth map, the sparse depth map does not record the depth information of each pixel point in the scene, but only records the depth values of some pixel points. Usually, the pixel points with depth values are relatively sparsely distributed. Obtain the first sparse depth map pixel-aligned with the first infrared speckle image, and the second sparse depth map pixel-aligned with the second infrared speckle image. Pixel alignment means that the pixels at the same position in the image correspond to the same position in the scene in different types of images (such as infrared speckle images and sparse depth maps), which is the basis for accurately filling the depth values subsequently.

[0080] Sparse depth maps have strong anti-interference ability for slight occlusion. Only when there is severe occlusion, the depth value will be 0. Therefore, for slightly occluded areas, after removing the incorrect depths on the dense depth map, the sparse depths at the corresponding positions are filled into the dense depth to filter out the incorrect depths caused by occlusion as much as possible while minimizing the impact on the filling rate of the depth map, greatly improving the depth quality of the depth camera under local occlusion of the lens.

[0081] Step S8: Remove the depth values of the occluded area in the first dense depth map, and fill the corresponding depth values in the first sparse depth map into the first dense depth map; remove the depth values of the occluded area in the second dense depth map, and fill the corresponding depth values in the second sparse depth map into the second dense depth map.

[0082] In this step, the depth information in the sparse depth map is used to correct the inaccurate depth values in the dense depth map due to occlusion, so as to improve the accuracy of the depth map. Since the occluded area has been marked in step S6, in the first dense depth map, the depth values of each pixel point in the occluded area are removed (they can be set to a specific invalid value, such as 0). The depth values at the corresponding positions in the first sparse depth map to the occluded area of the first dense depth map are filled into the corresponding positions of the first dense depth map. Because the sparse depth map records the depth information of some pixels, there may be more accurate depth values in the occluded area, and the quality of the first dense depth map can be improved through this filling operation.

[0083] In the same way as processing the first dense depth map, the depth values of the occluded area are removed in the second dense depth map, and then the depth values at the corresponding positions in the second sparse depth map are filled into the occluded area of the second dense depth map, thereby improving the accuracy of the second dense depth map.

[0084] Based on the detection of the occluded area in the previous step, this embodiment uses the sparse depth map to correct the depth values of the occluded area in the dense depth map, further optimizing the quality of the depth map and providing more accurate data for subsequent applications based on the depth map (such as 3D reconstruction, object detection, etc.).

[0085] In some embodiments, the time interval between the first moment and the second moment is inversely proportional to the maximum motion speed preset by the infrared speckle depth camera. In step S4, it is necessary to determine the motion state of the camera by comparing the dense depth maps at the first moment and the second moment. If the time interval is too long and the camera motion speed is fast, then the change of the scene may be very large, resulting in too large a difference between the images and depth maps at the first moment and the second moment, making it difficult to accurately determine whether the depth difference is caused by camera motion or the change of the objects in the scene itself. By making the time interval inversely proportional to the maximum motion speed, it can be ensured that the data at the two moments have good comparability under different motion speed conditions, improving the accuracy of camera motion state judgment. At the same time, step S6 determines the occlusion area based on the overlapping situation of the first alternative area and the second alternative area. If the time interval is not appropriate, it may cause large changes in the speckle pattern and depth information, resulting in inaccurate marking of the alternative area, thereby affecting the accuracy of occlusion area detection. Reasonably setting the time interval can reduce this influence and make the detection of the occlusion area more reliable.

[0086] Suppose in an industrial inspection scenario, the preset maximum motion speed v1 of the infrared speckle depth camera is relatively fast. According to the inverse ratio relationship, the set time interval t1 between the first moment and the second moment will be shorter, for example, 0.1 second. While in another indoor static scenario, the preset maximum motion speed v2 of the camera is slower, then the set time interval t2 can be appropriately increased, such as 1 second. In this way, the effectiveness and accuracy of the entire process can be ensured in different application scenarios.

[0087] Figure 3 This is a flowchart of the steps for marking the first alternative area in an embodiment of the present invention. As Figure 3 shown, the steps for marking the first alternative area in an embodiment of the present invention include:

[0088] Step S21: Extract the first speckle center on the first infrared speckle image.

[0089] In this step, the core position of each speckle in the first infrared speckle image is determined, preparing for subsequent analysis of the characteristics of the speckle area. The speckle center is a key feature point of the speckle pattern. There are various common extraction methods, such as the method based on gray value. Since the speckle usually appears as an area with relatively higher (or lower, depending on the imaging characteristics of the speckle) gray value in the image, the speckle center can be determined by finding the peak point of the local gray value. It is also possible to use the method of centroid calculation, taking the gray values of each pixel point in the speckle area as weights and calculating the centroid position of the area as the speckle center.

[0090] Step S22: Taking each of the first speckle centers as the center, a circular area with a radius of r is used as the neighborhood, and the gray value difference between the first speckle center and other pixels in the neighborhood is calculated to obtain the first difference.

[0091] In this step, by analyzing the gray value difference between the speckle center and its neighborhood pixels, the gray change situation of the speckle area is judged, providing a basis for marking possible occlusion areas.

[0092] Taking each of the extracted first speckle centers as the center, a radius r is set to form a circular neighborhood. The size of the radius r needs to be selected according to the actual situation, which determines the range of neighborhood pixels participating in the calculation. If r is too small, the gray change around the speckle may not be fully reflected; if r is too large, interference from too many irrelevant pixels may be introduced.

[0093] For each pixel in the neighborhood, the difference between its gray value and the gray value of the first speckle center is calculated. These differences can reflect the degree of gray difference between the speckle center and the surrounding pixels. Usually, some statistical processing is performed on these differences, such as calculating their average value, standard deviation, etc., and the obtained statistical result is used as the first difference.

[0094] Step S23: When the first difference is less than the first depth threshold, the speckle area where the first speckle center is located is marked as the first alternative area.

[0095] In this step, the first depth threshold is dynamically adjusted according to the average depth information and speckle density of the first infrared speckle image. In this step, according to the gray value difference between the speckle center and the neighborhood pixels, the speckle areas that may be occluded are screened out and marked as the first alternative areas. At the same time, considering the changes in image features in different scenarios, the judgment threshold is dynamically adjusted to improve the accuracy.

[0096] The calculated first difference is compared with the first depth threshold. If the first difference is less than the first depth threshold, it means that the gray change in the speckle area is small, and there may be situations such as occlusion. The speckle area where the first speckle center is located is marked as the first alternative area.

[0097] The first depth threshold is not fixed, but is dynamically adjusted according to the average depth information and speckle density of the first infrared speckle image. For example, when the average depth is large, the objects in the scene may be far from the camera, and the imaging characteristics of the speckles will change. At this time, the first depth threshold can be appropriately increased; when the speckle density is high, the mutual influence between the speckles may make the gray change more complex, and the first depth threshold also needs to be adjusted accordingly to ensure the accuracy of the marking. This dynamic adjustment mechanism can make the algorithm better adapt to different scenarios and image conditions.

[0098] Figure 4 This is a flowchart of the steps for judging the motion state in an embodiment of the present invention. As Figure 4 shown, the steps for judging the motion state in an embodiment of the present invention include:

[0099] Step S41: Subtract the first dense depth map and the second dense depth map pixel by pixel to obtain a third dense depth map, and filter to eliminate jumping noise.

[0100] In this step, by comparing the dense depth maps at two different times, the change of the scene depth is found, and at the same time, the abnormal depth changes caused by factors such as noise are removed to obtain a more reliable depth change map.

[0101] The first dense depth map and the second dense depth map are images that record the depth information of each pixel point in the scene, and the pixels of the two are aligned. Subtracting the pixel depth values at the corresponding positions of these two depth maps can obtain a new depth map, that is, the third dense depth map. Each pixel value in the third dense depth map represents the change amount of the depth of the pixel between the two times. For example, if the depth value of a certain pixel in the first dense depth map is d1 and the depth value of the corresponding pixel in the second dense depth map is d2, then the value of this pixel in the third dense depth map is d3 = d1 - d2.

[0102] In the actual process of collecting depth maps, it may be affected by factors such as sensor noise and environmental interference, resulting in some jumping abnormal depth values in the depth map. In order to eliminate the influence of these noises on subsequent analysis, it is necessary to perform filtering processing on the third dense depth map. Common filtering methods include median filtering, Gaussian filtering, etc. Median filtering is to sort the pixel values in the neighborhood and take the middle value as the new value of the current pixel, which can effectively remove salt-and-pepper noise; Gaussian filtering is to perform weighted averaging on the neighborhood pixels according to the Gaussian function, which can smooth the image and suppress Gaussian noise.

[0103] Step S42: Count the number of pixel points in the third dense depth map whose depth values exceed the second depth threshold, that is, the exceeding number.

[0104] In this step, the scale of the pixel points with large depth changes in the third dense depth map is quantified, which is used as a basis for judging whether the camera is moving. Set a second depth threshold, which is used to distinguish whether the depth change is caused by normal scene dynamics (such as the slight movement of small objects) or camera movement. Traverse each pixel in the third dense depth map, count the number of pixel points whose depth values are greater than the second depth threshold, and record this number as the exceeding number. For example, if the second depth threshold is Td and the depth value of a certain pixel in the third dense depth map is d, when d > Td, this pixel is included in the statistics of the exceeding number.

[0105] Step S43: If the exceeded quantity is greater than the first quantity threshold, it is considered that the infrared speckle depth camera is in a motion state, and step S5 is executed.

[0106] In this step, the first quantity threshold is adjusted according to the size and speckle spacing of the first infrared speckle pattern. Whether the camera is in a motion state is judged based on the number of pixel points with large depth changes, and at the same time, considering the influence of different image features on the judgment standard, the judgment threshold is dynamically adjusted to improve the accuracy.

[0107] The statistically obtained exceeded quantity is compared with the first quantity threshold. If the exceeded quantity is greater than the first quantity threshold, it indicates that between two moments, the depths of a large number of pixels in the scene have changed greatly. This large-scale depth change is very likely caused by the movement of the camera. Therefore, it is considered that the infrared speckle depth camera is in a motion state, and the subsequent step S5 is executed.

[0108] The first quantity threshold is not fixed, but is adjusted according to the size and speckle spacing of the first infrared speckle pattern. The larger the image size, the more pixel points there are, and the corresponding number of pixel points with normal depth changes that may appear will also increase. At this time, it is necessary to appropriately increase the first quantity threshold; the speckle spacing reflects the distribution density of the speckles. Different speckle spacings will result in different imaging characteristics and depth change laws of the speckles. Therefore, the first quantity threshold also needs to be adjusted according to the speckle spacing. Through this dynamic adjustment mechanism, the judgment of the camera's motion state can be made more accurate, adapting to different image conditions and application scenarios.

[0109] In some embodiments, in step S4, motion analysis is performed through the second infrared speckle images at multiple different second moments as the motion states of the multiple second infrared speckle patterns. Single judgment may be affected by accidental factors. For example, the sudden movement of an object in the scene at a certain moment may be misjudged as the movement of the camera. By introducing the second infrared speckle images at multiple different second moments for motion analysis, the motion state of the camera can be judged more comprehensively and accurately, reducing the possibility of misjudgment and improving the reliability of motion analysis.

[0110] Figure 5 It is a flowchart of steps for marking an occlusion area in an embodiment of the present invention. As Figure 5 shown, the steps for marking an occlusion area in an embodiment of the present invention include:

[0111] Step S61: Calculate the number of overlapping pixel points in the first alternative area and the second alternative area based on a fast overlapping calculation method based on an image mask.

[0112] In this step, accurately and efficiently determine the number of pixels in the overlapping part between the first alternative region and the second alternative region, providing a quantitative basis for subsequent judgment of whether it is an occluded region.

[0113] First, generate corresponding image masks for the first alternative region and the second alternative region respectively. An image mask is a binary image, where the pixel values within the alternative region are set to 1 (or 255), and the pixel values outside the region are set to 0. For example, if the first alternative region occupies certain pixel positions in the first infrared speckle image, in the corresponding mask image of the first alternative region, the pixel values at these positions are 1, and the rest are 0; similarly, generate the mask image of the second alternative region.

[0114] Multiply the two mask images pixel by pixel. Since the multiplication result is 1 only when the pixel values at the corresponding positions of the two mask images are both 1, the pixels with a value of 1 in the multiplied image represent the overlapping part of the two alternative regions. Finally, count the number of pixels with a value of 1 in the multiplied image to obtain the number of overlapping pixel points. This method based on image masks utilizes pixel-level operations, has a fast calculation speed, and is suitable for processing large-scale image data.

[0115] Step S62: If the number of overlapping pixel points exceeds the second quantity threshold, mark the first alternative region and the second alternative region as occluded regions; the second quantity threshold is adjusted according to the occlusion probability and the average size of the speckle region.

[0116] In this step, judge whether the first alternative region and the second alternative region are occluded regions according to the number of overlapping pixel points, and at the same time consider the differences in occlusion situations and speckle region characteristics in different scenarios, and dynamically adjust the judgment threshold to improve accuracy.

[0117] Compare the number of overlapping pixel points calculated in step S61 with the second quantity threshold. If the number of overlapping pixel points exceeds the second quantity threshold, it means that a large part of these two alternative regions coincide at two moments, and there is likely an occlusion situation. Mark the first alternative region and the second alternative region as occluded regions.

[0118] The second quantity threshold is not fixed, but is adjusted according to the occlusion probability and the average size of the speckle region. The occlusion probability reflects the likelihood of occlusion in the scene. If the occlusion probability is high, the second quantity threshold can be appropriately reduced to more sensitively detect occluded regions; the average size of the speckle region also affects the judgment criterion. When the average size is large, it may be necessary to increase the second quantity threshold to avoid misjudging normal speckle regions as occluded regions.

[0119] Step S63: Remove the non-corresponding closed regions in the first alternative region and the second alternative region to obtain the final occluded region.

[0120] In this step, the possibly mislabeled parts in the first alternative region and the second alternative region are removed, and the occluded region is further refined to obtain a more accurate final result.

[0121] In the first alternative region and the second alternative region, various closed regions are found through methods such as image segmentation or connected component analysis. A closed region refers to a region composed of continuous pixels and having an obvious boundary separation from the surrounding region.

[0122] Compare the closed regions in the first alternative region and the second alternative region, and remove those closed regions that have no corresponding relationship in the two regions. For example, if a certain closed region only exists in the first alternative region and has no corresponding overlapping part in the second alternative region, then this closed region is removed from the possible occluded regions. After such processing, the remaining region is the finally determined occluded region.

[0123] This embodiment realizes the accurate detection and refinement of the occluded region by accurately calculating the number of overlapping pixels, dynamically adjusting the judgment threshold, and removing mislabeled regions.

[0124] Figure 6 It is a schematic structural diagram of an infrared speckle depth camera occlusion detection system in an embodiment of the present invention. As Figure 6 shown, an infrared speckle depth camera occlusion detection system in an embodiment of the present invention includes:

[0125] A first acquisition module, configured to acquire a first infrared speckle image and a first dense depth map at a first moment; wherein, the first dense depth map and the first infrared speckle image are pixel-aligned;

[0126] A first difference module, configured to extract a first speckle center on the first infrared speckle image and calculate a first difference between the first speckle center and surrounding pixels; when the first difference is less than a first depth threshold, mark the speckle region where the first speckle center is located as the first alternative region;

[0127] A second acquisition module, configured to acquire a second infrared speckle image and a second dense depth map at a second moment; wherein, the second dense depth map and the second infrared speckle image are pixel-aligned;

[0128] A second difference module, configured to subtract the first dense depth map from the second dense depth map to obtain a third dense depth map. If the number of pixel points with a depth value exceeding a second depth threshold in the third dense depth map exceeds a first quantity threshold, it is considered that the infrared speckle depth camera is in a motion state, and the calculation module is executed;

[0129] A calculation module is used to extract the second speckle center on the second infrared speckle image and calculate the second difference between the second speckle center and surrounding pixels; when the second difference is less than the first depth threshold, mark the speckle area where the second speckle center is located as the second alternative area;

[0130] A marking module is used to mark the first alternative area and the second alternative area as occluded areas if the number of overlapping pixel points in the first alternative area and the second alternative area exceeds the second quantity threshold.

[0131] Specifically, the first acquisition module is responsible for acquiring the first infrared speckle image and the first dense depth map at the first moment, and ensuring that these two image data are aligned at the pixel level. This is the starting step of the entire occlusion detection process, providing initial image and depth information for subsequent analysis. The first infrared speckle image records the distribution of speckles in the scene at that moment, and the first dense depth map records the depth information corresponding to each pixel in the scene.

[0132] The first difference module processes the first infrared speckle image obtained by the first acquisition module. First, the first speckle center is extracted, which is a key step in identifying speckle features. Then, the first difference between the first speckle center and surrounding pixels is calculated. By comparing this difference with the first depth threshold, when the difference is less than the threshold, the speckle area where the first speckle center is located is marked as the first alternative area. The first alternative area is a preliminarily screened area where occlusion may exist.

[0133] The second acquisition module is similar to the first acquisition module, acquiring the second infrared speckle image and the second dense depth map at the second moment, and also ensuring pixel alignment between the two. The data at the second moment is used to compare with the data at the first moment to judge the motion state of the camera and further detect occluded areas.

[0134] The second difference module subtracts the first dense depth map obtained by the first acquisition module from the second dense depth map obtained by the second acquisition module to obtain the third dense depth map, which reflects the change in scene depth between the two moments. Then, the number of pixel points in the third dense depth map whose depth values exceed the second depth threshold is counted. If this number exceeds the first quantity threshold, it is judged that the infrared speckle depth camera is in a motion state and the calculation module is triggered.

[0135] After the second difference module judges that the camera is in a motion state, the calculation module processes the second infrared speckle image obtained by the second acquisition module. The second speckle center is extracted, the second difference between the second speckle center and surrounding pixels is calculated, and when the second difference is less than the first depth threshold, the speckle area where the second speckle center is located is marked as the second alternative area. The second alternative area is an area where occlusion may exist screened at the second moment.

[0136] The marking module compares the first alternative region marked by the first difference module and the second alternative region marked by the calculation module, and calculates the number of overlapping pixel points between them. If the number of overlapping pixel points exceeds the second quantity threshold, the first alternative region and the second alternative region are marked as occluded regions, completing the final occluded region detection.

[0137] The first acquisition module transfers the acquired first infrared speckle image and the first dense depth map to the first difference module and the second difference module. The first difference module uses the first infrared speckle image for speckle center extraction and difference calculation, while the second difference module performs depth comparison using the first dense depth map and the second dense depth map provided by the second acquisition module.

[0138] The second acquisition module transfers the second infrared speckle image and the second dense depth map to the second difference module and the calculation module. The second difference module is used to judge the camera motion state, and the calculation module marks the second alternative region based on the second infrared speckle image.

[0139] The data of the first alternative region marked by the first difference module and the second alternative region marked by the calculation module are transferred to the marking module for the final occluded region judgment.

[0140] The entire system starts with the first acquisition module. After completing data acquisition, it triggers the first difference module to mark the first alternative region.

[0141] The second acquisition module starts after the first acquisition module to obtain data at the second moment. Then, the second difference module judges the camera motion state based on the depth maps at two moments. If the camera is in a motion state, it triggers the calculation module to mark the second alternative region.

[0142] When both the first alternative region and the second alternative region are marked, it triggers the marking module to perform the final occluded region marking.

[0143] Based on pixel-level analysis, this embodiment acquires infrared speckle images and dense depth maps at different moments respectively. By calculating the differences between the speckle center and surrounding pixels, it accurately locates possible occluded regions to achieve high-precision detection. At the same time, by comparing the dense depth maps at different moments, it judges the camera motion state to avoid misjudgment caused by camera movement, making the detection results more reliable. By comprehensively using multi-frame information, it not only analyzes single-frame images but also determines the occluded region by comparing the number of overlapping pixel points in the alternative regions of two frames, enhancing the stability and accuracy of the detection, reducing noise interference, and being able to complete the detection in a short time with good real-time performance, which is suitable for various application scenarios requiring real-time feedback.

[0144] In an embodiment of the present invention, there is also provided an infrared speckle depth camera occlusion detection device, including a processor and a memory that stores executable instructions of the processor. The processor is configured to execute the steps of an infrared speckle depth camera occlusion detection method by executing the executable instructions.

[0145] As described above, based on pixel-level analysis, this embodiment respectively acquires infrared speckle images and dense depth maps at different times, accurately locates possible occlusion areas by calculating the differences between the speckle centers and surrounding pixels, and achieves high-precision detection. At the same time, by comparing the dense depth maps at different times, the camera motion state is judged to avoid misjudgment caused by camera movement, making the detection result more reliable. By comprehensively using multi-frame information, not only single-frame images are analyzed, but also the occlusion area is determined by comparing the number of overlapping pixel points in the alternative areas of two frames, enhancing the stability and accuracy of the detection and reducing noise interference. And it can complete the detection in a short time, with good real-time performance, and is applicable to various application scenarios that require real-time feedback.

[0146] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "platform" here.

[0147] Figure 7 is a schematic structural diagram of an infrared speckle depth camera occlusion detection device in an embodiment of the present invention. The following will refer to Figure 7 to describe the electronic device 600 according to this embodiment of the present invention. Figure 7 The electronic device 600 shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present invention.

[0148] As Figure 7 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0149] Among them, the storage unit stores program codes, and the program codes can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above-mentioned part of the infrared speckle depth camera occlusion detection method of this specification. For example, the processing unit 610 can execute the steps as Figure 1 shown.

[0150] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.

[0151] The storage unit 620 may also include a program / utilities 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a grid environment.

[0152] The bus 630 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0153] The electronic device 600 may also communicate with one or more external devices 700 (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or may communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be through an input / output (I / O) interface 650. Moreover, the electronic device 600 may also communicate with one or more grids (such as a local area network (LAN), a wide area network (WAN), and / or a public grid, such as the Internet) through a grid adapter 660. The grid adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Figure 7 not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0154] An embodiment of the present invention also provides a computer-readable storage medium for storing a program, and the steps of an infrared speckle depth camera occlusion detection method are implemented when the program is executed. In some possible implementation manners, various aspects of the present invention may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above-mentioned part of the infrared speckle depth camera occlusion detection method of this specification.

[0155] As shown above, this embodiment is based on pixel-level analysis. Infrared speckle images and dense depth maps are obtained at different times respectively. By calculating the difference between the speckle center and surrounding pixels, possible occlusion areas are accurately located to achieve high-precision detection. At the same time, the camera motion state is judged by comparing the dense depth maps at different times to avoid misjudgment caused by camera movement, making the detection result more reliable. By comprehensively using multi-frame information, not only single-frame images are analyzed, but also the occlusion area is determined by comparing the number of overlapping pixel points in the alternative areas of two frames, enhancing the stability and accuracy of the detection and reducing noise interference. And it can complete the detection in a short time, with good real-time performance, and is applicable to various application scenarios that require real-time feedback.

[0156] Figure 8 It is a schematic structural diagram of a computer-readable storage medium in an embodiment of the present invention. Refer to Figure 8 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described. It can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0157] The program product can adopt any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0158] The computer-readable storage medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program used by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0159] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0160] This embodiment is based on pixel-level analysis. At different times, an infrared speckle image and a dense depth map are respectively obtained. By calculating the difference between the speckle center and the surrounding pixels, the possible occlusion areas are accurately located to achieve high-precision detection. At the same time, by comparing the dense depth maps at different times, the camera motion state is judged to avoid misjudgment caused by camera movement, making the detection result more reliable. By comprehensively using multi-frame information, not only is a single-frame image analyzed, but also the occlusion area is determined by comparing the number of overlapping pixel points in the alternative areas in two frames, enhancing the stability and accuracy of the detection and reducing noise interference. And it can complete the detection in a short time, with good real-time performance, and is applicable to various application scenarios that require real-time feedback.

[0161] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0162] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for detecting occlusion of an infrared speckle depth camera, characterized in that: include: Step S1: acquiring a first infrared speckle image and a first dense depth map at a first moment; wherein the first dense depth map and the first infrared speckle image are pixel-aligned; Step S2: extracting a first speckle center from the first infrared speckle image, and calculating a first difference between the first speckle center and surrounding pixels; when the first difference is less than a first depth threshold, marking a speckle region where the first speckle center is located as a first candidate region; Step S3: Acquire a second infrared speckle image and a second dense depth map at a second moment; wherein the pixels of the second dense depth map and the second infrared speckle image are aligned; Step S4: subtract the first dense depth map from the second dense depth map to obtain a third dense depth map. If the number of pixel points whose depth values ​​exceed the second depth threshold in the third dense depth map exceeds the first number threshold, it is considered that the infrared speckle depth camera is in motion, and step S5 is executed; Step S5: extracting a second speckle center from the second infrared speckle image, and calculating a second difference between the second speckle center and surrounding pixels; when the second difference is less than a first depth threshold, marking a speckle region where the second speckle center is located as a second candidate region; Step S6: if the number of overlapping pixels in the first candidate area and the second candidate area exceeds a second number threshold, marking the first candidate area and the second candidate area as occluded areas.

2. The infrared speckle depth camera occlusion detection method according to claim 1, characterized in that: Also includes: Step S7: acquiring a first sparse depth map of the first infrared speckle image and a second sparse depth map of the second infrared speckle image; wherein the first sparse depth map and the first infrared speckle image are pixel-aligned, and the second sparse depth map and the second infrared speckle image are pixel-aligned; Step S8: removing the depth value of the occluded area in the first dense depth map, and filling the corresponding depth value in the first sparse depth map into the first dense depth map; removing the depth value of the occluded area in the second dense depth map, and filling the corresponding depth value in the second sparse depth map into the second dense depth map.

3. The infrared speckle depth camera occlusion detection method according to claim 1, characterized in that: The time interval between the first moment and the second moment is inversely proportional to a preset maximum movement speed of the infrared speckle depth camera.

4. The infrared speckle depth camera occlusion detection method according to claim 1, characterized in that: Step S2 includes: Step S21: extracting a first speckle center from the first infrared speckle image; Step S22: taking each of the first speckle centers as the center of the circle and a circular area with a radius r as the neighborhood, calculating the grayscale value difference between the first speckle center and other pixels in the neighborhood to obtain a first difference; Step S23: when the first difference is less than a first depth threshold, marking the speckle region where the first speckle center is located as a first candidate region; wherein the first depth threshold is dynamically adjusted according to average depth information and speckle density of the first infrared speckle image.

5. The infrared speckle depth camera occlusion detection method according to claim 1, characterized in that: Step S4 includes: Step S41: subtracting the first dense depth map from the second dense depth map pixel by pixel to obtain a third dense depth map, and filtering to eliminate jumping noise; Step S42: Counting the number of pixels in the third dense depth map whose depth values ​​exceed the second depth threshold, that is, the excess number; Step S43: if the excess quantity is greater than a first quantity threshold, it is considered that the infrared speckle depth camera is in motion, and step S5 is executed; wherein the first quantity threshold is adjusted according to the size of the first infrared speckle pattern and the speckle spacing.

6. The infrared speckle depth camera occlusion detection method according to claim 1, characterized in that: In step S4, motion analysis is performed on a plurality of the second infrared speckle images at different second moments as motion states of a plurality of the second infrared speckle patterns.

7. The infrared speckle depth camera occlusion detection method according to claim 1, characterized in that: Step S6 includes: Step S61: Calculating the number of overlapping pixels in the first candidate area and the second candidate area using a fast overlap calculation method based on an image mask; Step S62: if the number of overlapping pixels exceeds a second number threshold, marking the first candidate area and the second candidate area as occlusion areas; the second number threshold is adjusted according to the occlusion probability and the average size of the speckle area; Step S63: Eliminate the non-corresponding closed areas in the first candidate area and the second candidate area to obtain a final occluded area.

8. An infrared speckle depth camera occlusion detection system, used to implement the infrared speckle depth camera occlusion detection method according to any one of claims 1 to 7, characterized in that: include: A first acquisition module, configured to acquire a first infrared speckle image and a first dense depth map at a first moment; wherein the first dense depth map and the first infrared speckle image are pixel-aligned; a first difference module, configured to extract a first speckle center from the first infrared speckle image, and calculate a first difference between the first speckle center and surrounding pixels; when the first difference is less than a first depth threshold, marking a speckle region where the first speckle center is located as a first candidate region; A second acquisition module, configured to acquire a second infrared speckle image and a second dense depth map at a second moment; wherein the second dense depth map and the second infrared speckle image are pixel-aligned; a second difference module, configured to subtract the first dense depth map from the second dense depth map to obtain a third dense depth map; if the number of pixels whose depth values ​​exceed the second depth threshold in the third dense depth map exceeds the first number threshold, the infrared speckle depth camera is considered to be in motion, and a calculation module is executed; a calculation module, configured to extract a second speckle center from the second infrared speckle image, and calculate a second difference between the second speckle center and surrounding pixels; when the second difference is less than a first depth threshold, mark a speckle region where the second speckle center is located as a second candidate region; A marking module is used to mark the first candidate area and the second candidate area as occluded areas if the number of overlapping pixels in the first candidate area and the second candidate area exceeds a second number threshold.

9. An infrared speckle depth camera occlusion detection device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the infrared speckle depth camera occlusion detection method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the infrared speckle depth camera occlusion detection method described in any one of claims 1 to 7 are implemented.

Images