Three-dimensional structured light camera device with light source and method thereof

Abstract

A three-dimensional structured light camera device (310) includes a LUT memory (307) and a processor (306) configured to: perform object detection and identify a 3D object region of interest of a target speckle image; and executing a local adaptive binarization process, wherein the local adaptive binarization process searches the speckle characteristics of the stored reference speckle image in the LUT memory and matches the speckle characteristics of the target speckle image with the searched speckle characteristics of the reference speckle image. In some examples, the processor (306) may perform a morphological operation on matching speckle features of the target speckle image, the morphological operation producing a noise-reduced target speckle image; and performing binary coding compression. Depth value calculation of a 3D target object is performed in response to performing the binary coding compression on the noise-reduced target speckle image. In this manner, speckle matching may be accelerated by reducing the number of computations.

Description

Technical Field

[0001] The field of the invention relates to a three-dimensional (3D) structured light camera apparatus having a light source and a method of processing depth information using speckle depth value calculation for 3D modeling of a target object. Background Art

[0002] In recent years, three-dimensional (3D) structured light cameras have become increasingly popular and are now widely used in a variety of smart products, such as mobile phones, robots, smart homes, smart security, vehicles and other fields. 3D structured light cameras are sometimes also called 'depth cameras', which are different from standard cameras. The difference between 3D structured light cameras and 3D light cameras is that a special speckle pattern is projected onto the object. This speckle pattern is pseudo-random, and each speckle distributed in space has a unique code that can be used to identify the speckle, thereby achieving depth ranging. In addition to capturing two-dimensional picture information, 3D structured light cameras can also obtain three-dimensional distance information, usually through a speckle depth decoding algorithm that can be used for 3D modeling of the target. In this context, speckles are regarded as small marks, spots or spots in the image.

[0003] Figure 1 A simplified known diagram 100 of a 3D structured light camera 110 arrangement configured to take a picture of a 3D object 130 is shown. The 3D structured light camera 110 comprises a speckle emitter 120, an infrared (IR) light emitting diode (LED) 122 and an IR camera 124. The IR camera 124 has only one complete reference speckle image, which is initially stored in the camera memory by the camera vendor. Projecting narrow-band light onto a three-dimensionally shaped surface produces illumination rays that appear distorted from angles other than the projector and can be used for geometric reconstruction of the surface shape. The projection method typically uses incoherent light and works essentially like a video projector. The pattern is typically generated by passing the light through, say, a digital spatial light modulator, which is typically based on one of the three most widely used digital projection technologies currently: a transmissive liquid crystal, a reflective liquid crystal on silicon (LCOS) or a digital light processing (DLP; moving micro-mirrors) modulator.

[0004] Figure 2A known arrangement 200 provided by a module manufacturer is shown, including an embedded application specific integrated circuit (ASIC) designed to include a depth decoding algorithm in a camera module. The known arrangement 200 includes a 3D structured light camera 110 with an additional embedded application specific integrated circuit (ASIC) configured to generate IR and depth frames 210 and a 3D structured light camera 110 without an additional embedded ASIC configured to generate IR and speckle frames 230. The IR and depth frames 210 are provided to a main computer unit (MCU) system on chip (SOC) 220, which is configured to run a vision algorithm 222 from an algorithm library. The IR and speckle frames 230 are provided to a main computer unit (MCU) system on chip (SOC) 240, which is configured to run a vision algorithm 242 and a depth decoding algorithm 244 from an algorithm library. The main goal of these two paths is to obtain IR and depth images of a target. For the IR and depth frame 210 path (110-210-220), the IR and depth images of a target can be obtained directly through the object detection algorithm of the vision algorithm 222. For the IR and speckle frame 230 path (110-230-240), the IR and speckle images of the target can only be obtained through the object detection algorithm 242, and then the process still needs to use the depth decoding algorithm to obtain the target depth image. The known arrangement 200 then provides an output 250 of the IR and depth images of the target, which can be used for many 3D solutions, such as 3D modeling, anti-fraud, object positioning, volume measurement, AR / VR, etc.

[0005] Depth estimation based on 3D structured light laser speckle camera is an active 3D depth measurement method. It is well known that speckle structure has two characteristics that provide conditions for depth measurement. The first characteristic is that speckle has strong autocorrelation and weak correlation, which means that each speckle is distinguishable. The second characteristic is that the speckle structure is translation invariant, that is, in an ideal speckle structured light system, the speckle pattern moves as a whole at different distances, so that the size and distribution of the speckle does not change with the change of distance. Most existing depth decoding algorithms indiscriminately perform depth estimation for each pixel of each frame. In addition, these algorithms do not optimize and accelerate the speckle matching process, so they inherently require / cause a lot of calculations. Therefore, module manufacturers embed application-specific integrated circuits (ASICs) designed specifically to run depth decoding algorithms within camera modules. However, the additional ASIC will bring the problem of high power consumption, resulting in a high cost of 3D structured light cameras, which discourages customers who like to use 3D structured light cameras but are price-sensitive.

[0006] A paper co-authored by Gu Jiawei, Xie Xiaopeng, Cao Yibo, and Liu Haoxin, entitled 'A Semi-Dense Depth Map Acquisition Algorithm Based on Laser Speckle [J]' and published in the Chinese Journal of Lasers (2020, 47(3):0304004), describes a speckle extraction method using local adaptive binarization. The inventors have also recognized and understood that the method adopted in this paper proposes to compress memory by sacrificing the resolution of the speckle image, so the matching accuracy cannot reach the pixel level. Other known technologies have limitations on the supported platforms. The inventors have recognized and understood that it may be useful to design a method that can be independent of the ASIC architecture and run on a main computer unit (MCU) platform, as well as suitable artificial intelligence (AI) / machine language (ML) algorithms in some cases, to make the method cost-effective. The inventors have also recognized and understood that it may be useful to design a method that balances accuracy, speed, and computational efficiency with respect to memory usage. Therefore, there is a need for an apparatus having a light source and a method for performing depth value calculation. Summary of the invention

[0007] According to a first aspect of the present invention, there is provided a three-dimensional 3D structured light camera device, comprising:

[0008] an infrared IR light source configured to illuminate a three-dimensional 3D target object, a speckle emitter, and an IR camera;

[0009] a lookup table (LUT) memory configured to store a reference speckle image of the 3D target object; and

[0010] a processor operably coupled to the light source and the LUT memory and configured to:

[0011] performing object detection on the target speckle image;

[0012] Identifying a 3D target object focus area of ​​the target speckle image;

[0013] performing a local adaptive binarization process, the local adaptive binarization process searching for speckle features of a stored reference speckle image in the LUT memory and matching the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image;

[0014] searching for matching speckle features of the target speckle image; and

[0015] Depth value calculation of the 3D target object is performed in response to the target speckle image.

[0016] In one or more embodiments, in response to the processor being configured to find the matching speckle feature of the target speckle image, the processor is further configured to:

[0017] performing a morphological operation on the matching speckle features of the stored reference speckle image and the target speckle image, the morphological operation producing a noise-reduced target speckle image;

[0018] performing binary coding compression on the stored reference speckle image and the noise-reduced target speckle image; and

[0019] Depth value calculation of the 3D target object is performed in response to performing the binary coding compression on the noise-reduced target speckle image.

[0020] In one or more embodiments, the processor being configured to perform binary encoding compression on the noise-reduced target speckle image includes the processor being configured to compress an eight-pixel image into one-byte image data.

[0021] In one or more embodiments, the LUT memory is configured to store at least one byte of image data for eight pixel speckle features of the stored reference speckle image.

[0022] In one or more embodiments, the processor is configured to match the one-byte speckle feature of the target speckle image with the searched speckle feature of the stored reference speckle image.

[0023] In one or more embodiments, the processor is configured to perform object detection on the target speckle image, including: the processor is configured to obtain one or more coordinate parameters of the target bounding box, and the processor is configured to adaptively extract the 3D target object focus area from the one or more coordinate parameters for processing.

[0024] In one or more embodiments, the processor is configured to convert the target speckle image and the reference speckle image from original red-green-blue RGB speckle images into single-channel grayscale speckle images with pixel values ​​ranging from '0' to '255'.

[0025] In one or more embodiments, the processor being configured to perform local adaptive binarization includes: the processor being configured to separate background information and speckle of the target speckle image.

[0026] In one or more embodiments, the processor being configured to perform local adaptive binarization includes the processor being configured to compare and filter the target speckle image using a threshold applied within a set window size.

[0027] In one or more embodiments, the processor applies the following equation:

[0028]

[0029] in:

[0030] I(x,y) indicates the grayscale value of any pixel in the original image;

[0031] I′(x,y) is the pixel value after local adaptive binarization; and

[0032] T(x,y) is the local adaptive binarization threshold.

[0033] In one or more embodiments, the local adaptive binarization threshold is an average value of a plurality of pixels in the window, where the grayscale pixel value I(x,y) is identified as the center of the window and r is identified as the radius of the window.

[0034] In one or more embodiments, the processor is configured to first train the speckle matching process for the camera image by first being configured to create a reference speckle image.

[0035] According to a second aspect of the present invention, there is provided an integrated circuit for a 3D structured light camera apparatus, the integrated circuit comprising a processor according to any of the descriptions disclosed herein.

[0036] According to a third aspect of the present invention, there is provided a method for performing depth value calculation in a three-dimensional 3D structured light camera device, the method comprising:

[0037] Illuminating a three-dimensional 3D target object;

[0038] storing a reference speckle image of the 3D target object;

[0039] performing object detection on the target speckle image of the 3D target object;

[0040] Identifying a 3D target object focus area of ​​the target speckle image;

[0041] performing a local adaptive binarization process, the local adaptive binarization process comprising searching for speckle features of a stored reference speckle image in a lookup table (LUT) memory and matching the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image;

[0042] searching for matching speckle features of the target speckle image; and

[0043] Depth value calculation of the 3D target object is performed in response to the target speckle image.

[0044] In one or more embodiments, in response to the processor finding the matching speckle feature of the target speckle image, the method includes:

[0045] performing a morphological operation on the matched speckle features of the target speckle image, the morphological operation generating a target speckle image with reduced noise;

[0046] performing binary coding compression on the noise-reduced target speckle image; and

[0047] Depth value calculation of the 3D target object is performed in response to performing the binary coding compression on the noise-reduced target speckle image.

[0048] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Additional details, aspects and embodiments will be described by way of example only with reference to the drawings. In the drawings, similar reference numerals are used to identify similar or functionally similar elements. The elements in the drawings are shown for simplicity and clarity, and these elements are not necessarily drawn to scale.

[0050] Figure 1 A simplified known diagram of a 3D structured light camera is shown.

[0051] Figure 2 A known arrangement provided by a module manufacturer is shown, comprising an embedded application specific integrated circuit (ASIC) designed to include a depth decoding algorithm in the camera module.

[0052] Figure 3 An example of the information flow according to some example depth value calculations is shown.

[0053] Figure 4 An example of a high-level flowchart of depth value calculation according to some examples is shown.

[0054] Figure 5 An example image region after local adaptive binarization according to some examples and an image region after a morphological opening operation to remove small noise data of the binarized image are shown.

[0055] Figure 6 An example schematic diagram of a speckle matching process according to some examples is shown.

[0056] Figure 7 An example of a table representing the number of bits that are '1' in an 8-bit binary value is shown, according to some examples.

[0057] Figure 8 An example of a high-level flow chart of a speckle matching and search process according to some examples is shown.

[0058] Fig. 9 An example of an exemplary image before and after sub-pixel level processing according to some examples is shown. DETAILED DESCRIPTION

[0059] The examples described herein propose an apparatus with a light source and a method for performing very efficient and cost-effective depth value calculation based on an infrared speckle image of a 3D structured light camera. One aspect of the proposed method is configured to adaptively process a target region of interest according to the requirements of the depth value calculation, which avoids a large amount of computing resources being wasted on an invalid region or a region without a target frame of interest. It should be noted that adaptively processing the target region of interest is a new operation for depth decoding, especially for application scenarios with specific targets. In some examples, the specific target may be an object of processing application interest, such as the target of face recognition is to process a face, the target of gesture recognition and tracking is to process, for example, a hand, a person detection subject, etc. The examples described herein then propose local adaptive binarization of a reference speckle image and a target speckle image to extract complete and clear speckle features and perform binary coding compression, which can save computing resources. Thereafter, a morphological operation is performed on the target speckle image to remove noise points, thereby ensuring the accuracy when extracting speckle features, because the data after noise removal will obtain a higher speckle matching accuracy.

[0060] Some of the examples described herein also suggest that binary encoding can be used to compress the memory space required for depth value calculation and processing by, for example, up to 8 times. Under the premise of ensuring that the accuracy of depth disparity information is not reduced, the speckle matching calculation between the reference speckle image and the target speckle image can be optimized and accelerated by, for example, more than 12 times through an efficient search strategy and a lookup table (LUT) method. Some of the examples described herein also suggest that for distant objects, sub-pixel level (SPL) processing can be provided to further improve the accuracy of depth estimation by 2 times. Finally, in some of the examples described herein, the actual depth distance value can be calculated based on the speckle projection principle of 3D structured light. In this way, some of the examples described herein do not require additional embedded chips (e.g., application-specific integrated circuits (ASICs)) specifically for applying depth decoding algorithms, thereby helping to reduce costs.

[0061] Compared to the teaching of 'A semi-dense depth map acquisition algorithm based on laser speckle [J]' that only uses local adaptive binarization processing when extracting speckle, the example described in this article uses morphological operations to remove noise points after local adaptive binarization processing. In addition, compared with the teaching of 'A semi-dense depth map acquisition algorithm based on laser speckle [J]', the memory is compressed by sacrificing the resolution of the speckle image. In contrast, the example described in this article uses compression and optimization techniques that compress the space by 8 times without losing any original information or sacrificing the resolution of the speckle image. The example described in this article also optimizes the data storage space, achieves pixel-level matching accuracy, and provides accuracy optimization for long-distance data.

[0062] A first aspect described herein provides a three-dimensional 3D structured light camera apparatus for processing and depth value calculation, comprising: an infrared IR light source configured to illuminate a three-dimensional 3D target object, a speckle emitter and an IR camera. A lookup table LUT memory is configured to store a reference speckle image of the 3D target object; and a processor is operably coupled to the light source and the LUT memory and is configured to: perform object detection on the target speckle image; identify a 3D target object interest region of the target speckle image; perform a local adaptive binarization process, the local adaptive binarization process searches for speckle features of the stored reference speckle image in the LUT memory and matches the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image; find matching speckle features of the target speckle image; perform morphological operations on the matching speckle features of the target speckle image, the morphological operations generating a noise-reduced target speckle image; perform binary coding compression on the noise-reduced target speckle image; and perform depth value calculation of the 3D target object in response to performing binary coding compression on the noise-reduced target speckle image.

[0063] In this way, the examples described herein can optimize and accelerate the speckle matching process with a relatively small number of computations by adaptively extracting the region of interest of the object and then using an efficient search strategy using a lookup table. In addition, module manufacturers can embed an application specific integrated circuit (ASIC) that is specifically designed to run the depth decoding algorithm within the camera module while reducing power consumption, thereby reducing the cost of the 3D structured light camera. Specifically, the examples described herein can perform pre-processing using local adaptive binarization and morphological operations to extract complete and clear speckle features.

[0064] In some examples, the processor may perform binary coded compression on the noise-reduced target speckle image, which may include the processor being configured to compress eight pixels of the image into one byte of image data. In this way, the binary coded compression stores eight pixels in one byte without losing information for speckle matching. In some examples, the lookup table memory may be configured to store at least one byte of image data of eight pixel speckle features of the stored reference speckle image. In this way, using the LUT reduces the computational complexity of the matching process. In some examples, the processor may be configured to match at least one byte of speckle features of the target speckle image with the searched speckle features of the stored reference speckle image. In this way, the storage and search strategy of the image data may be more easily implemented.

[0065] In some examples, the processor may be configured to perform object detection on the target speckle image, which may include the processor being configured to obtain one or more coordinate parameters of the target bounding box, and the processor being configured to adaptively extract the 3D target object region of interest from the one or more coordinate parameters for processing. In this way, the processor may target the speckle features of interest rather than obtaining the speckle features from the entire object region including a plurality of speckles of no interest (thereby wasting precious processing resources).

[0066] In some examples, the processor may be configured to convert the target speckle image and the reference speckle image from raw red-green-blue RGB speckle images into single-channel grayscale speckle images with pixel values ​​ranging from '0' to '255'.

[0067] In some examples, the processor may be configured to first train a speckle decoding process for the camera images by first being configured to create a reference speckle image to be matched with the object speckle image. In the context of the examples described herein, the expression 'speckle decoding process' encompasses decoding speckle information of a target object / image into depth information, wherein speckle matching is considered as a part of the speckle decoding process.

[0068] In some examples, the processor may be configured to convert the target speckle image and the reference speckle image from raw red-green-blue RGB speckle images into single channel grayscale speckle images with pixel values ​​ranging from '0' to '255'. In some examples, the processor may be configured to separate background information and speckle of the target speckle image. In some examples, the processor may be configured to compare and filter the target speckle image using a threshold applied within a set window size. In some examples, the processor may be configured to first train the speckle decoding process for the camera image by first being configured to create a reference speckle image.

[0069] The reference speckle frame 320 initially has only one image. In some examples, the entire reference speckle image originally stored in the camera memory by the camera vendor is replaced by the processed entire reference speckle image 330 after local adaptive binarization, morphological operations and / or binary coded compression.

[0070] In some examples, an integrated circuit for a 3D structured light camera device includes the above processor.

[0071] A second aspect described herein discloses a method for performing depth value calculation in a three-dimensional 3D structured light camera device. The method includes: illuminating a three-dimensional 3D target object, storing a reference speckle image of the 3D target object; performing object detection on a target speckle image of the 3D target object; identifying a 3D target object interest region of the target speckle image; performing a local adaptive binarization process, the local adaptive binarization process including searching for speckle features of the stored reference speckle image in a lookup table LUT memory and matching the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image; searching for matching speckle features of the target speckle image; performing a morphological operation on the matching speckle features of the target speckle image, the morphological operation generating a noise-reduced target speckle image; performing binary coding compression on the noise-reduced target speckle image; and performing depth value calculation of the 3D target object in response to performing binary coding compression on the noise-reduced target speckle image.

[0072] In this way, an improved way of performing depth value calculations can be achieved, such as requiring less calculations, reducing storage space, etc. Thereafter, the depth calculation value can then be used for a range of applications, such as the depth calculation value can be combined with two-dimensional (2D) information to perform 3D modeling. The depth calculation value can be used in the field of face recognition, where 3D faces are not affected by natural light. This method can achieve perfect segmentation even if the foreground and background colors are similar, and its performance under complex head postures is far better than using 2D faces purely. At the same time, 3D face recognition is truly secure face recognition, which can effectively avoid security attacks, such as photos / screen pictures, face-changing algorithms, wearing masks / 3D face models, etc. In addition, it can be imagined that the depth calculation value may be very suitable for anti-fraud. In addition, it can be imagined that the depth value can also be used in a variety of applications, such as intelligent human-computer interaction, three-dimensional reconstruction, and autonomous robot navigation.

[0073] Figure 3An example of a speckle structure depth estimation 300 employed by a 3D structured light speckle camera 310 according to some examples is shown. The 3D structured light speckle camera 310 includes a speckle emitter 302, an infrared (IR) light emitting diode (LED) 304, a processor 306, a lookup table (LUT) memory 307, and an IR camera 308. According to some examples, the depth estimation 300 employed by the processor 306 in the 3D structured light speckle camera 310 is configured to adaptively extract and process only a (smaller) target region of interest from the obtained 3D depth information, in contrast to known depth decoding algorithms that process all depth information and indiscriminately produce a depth estimate for each pixel of each frame without considering that computational resources may be wasted on many invalid regions and non-target frames.

[0074] In the depth estimation 300, the first operation is to train 312 a speckle decoding process, for example once, by a processor in the 3D structured light speckle camera 310 with reference to a speckle frame 322 at 320. The reference speckle frame is captured based on a white clear background without a 3D target object. In some examples, the speckle distribution on the reference speckle frame will be used as a standard for comparison with the target speckle frame. If a 3D object is present, the speckle spots of the object in the target speckle image will be offset relative to the standard. Thereafter, a speckle preprocessing operation is performed at 324. Here, the speckle preprocessing operation is configured to extract and compress speckle features, and in some examples includes one or more of the following operations: local adaptive binarization, morphological operations, binary coding compression, so as to obtain a preprocessed reference speckle frame 332 at 330. Thereafter, at 334, the target sliding area of ​​the preprocessed reference speckle frame 332 is cropped out, so as to obtain a preprocessed target template image 342 at 340. In the context of the examples described herein, the pre-processed target template image comprises a target-related reference speckle image compressed using speckle feature extraction and binary coding. In the context of the examples described herein, the reference speckle frame is provided by a 3D camera vendor, and the reference speckle image comprises a target-related reference speckle image that can be cropped from the reference speckle frame according to target coordinates.

[0075] Once the training operation is performed, an object speckle frame 352 is received at 350. At 354, a target region of the object speckle frame 352 is extracted / cropped out to obtain a target speckle image 362 at 360. At this point, a speckle pre-processing operation is performed at 364. Figure 3As shown, the reference speckle frame is known from the beginning and does not change, so in some examples the reference speckle frame is processed first. If a target is detected, the target is cropped out from the pre-processed reference speckle frame to obtain a pre-processed target-related reference speckle image. Here, in this example, the speckle pre-processing operation is performed only on the object speckle frame to obtain a pre-processed target image 372 at 370. In addition, the processor 306 of the IR camera 308 replaces the entire original reference speckle image stored in the memory of the IR camera 308 by the camera vendor with the processed entire reference speckle image after local adaptive binarization, morphological operations and / or binary coding compression. Thereafter, at 380, a speckle matching process is performed to obtain a target depth disparity map 392 at 390. Therefore, the flowchart 300 includes two pipelines, the first pipeline focuses on obtaining a pre-processed target-related reference speckle image, and this is only performed once for each camera, while the second pipeline focuses on obtaining a pre-processed target object speckle image when a target is detected, and then speckle matching is performed on the two images. Speckle matching will help each pixel on the target speckle image find a matching reference pixel position. Finally, a depth disparity map of the target object can be obtained.

[0076] It is contemplated that in some examples, the processor 306 may be employed in an integrated circuit for a 3D structured light camera device. Although an example is described with respect to using a training operation to obtain a pre-processed reference speckle frame 332, it is contemplated that in other examples, the reference speckle image may be preloaded or received separately.

[0077] Adaptively extracting regions of interest

[0078] Reference now Figure 4 , shows an example of a high-level flow diagram of depth estimation / decoding 400 according to some examples. At 410, a processor (e.g. Figure 3 The processor 306 in Figure 3 The operation of the 3D structured light speckle camera 310 in the image processing apparatus obtains or receives the IR frame and the speckle frame. Depth estimation based on the 3D structured light speckle camera is an active three-dimensional depth measurement method. As is known to all, there are two characteristics of the speckle image structure that provide conditions for depth measurement. The first characteristic is that the speckle exhibits both strong autocorrelation and weak correlation, which means that each speckle is distinguishable. The second characteristic is that the speckle structure has translation invariance, that is, in an ideal speckle structured light system, the speckle pattern moves as a whole at different distances, so that the size and distribution of the speckle does not change according to the change of distance.

[0079] Speckle feature extraction preprocessing and binary coding compression

[0080] According to some examples, and because in one example the reference speckle image is Figure 3 The reference speckle image 312 is pre-stored, and its ambient brightness and background information are different from the target speckle image 362 captured in real time. In order to ensure the matching accuracy of the reference speckle and the target speckle, according to the example described in this article, the interference information is removed so that a complete and clear speckle feature image can be extracted.

[0081] Therefore, at 420, the flowchart 400 for depth estimation / decoding then utilizes object detection to adaptively extract the region of interest. In one example, the 3D structured light speckle camera will alternately output a pure infrared image and an infrared speckle image projected using laser speckle. When the pure infrared image and the infrared speckle image are obtained, an object detection algorithm may be first applied to the pure infrared image to obtain the coordinates of the target bounding box. In the example described herein, the pure infrared frame is captured by an IR camera without projecting a speckle pattern. Laser speckle is a speckle pattern projected by a speckle emitter. For example, a processor (e.g., Figure 3 The processor 306 in the image processing unit 306 or the algorithm used by the processor may be configured to provide an input interface of, for example, one or more coordinate parameters, which may, for example, adaptively crop / extract the region of interest for processing according to any product requirements. Therefore, at 430, a speckle feature extraction preprocessing operation is performed, such as by local adaptive binarization and morphological operations, and at 440, a binary coding compression is performed to obtain complete and clear speckle features.

[0082] First, in one example of 430 and 440, the original speckle image may be received as a grayscale or converted to a grayscale, so that the original red-green-blue (RGB) speckle image becomes a single-channel grayscale speckle image with a pixel value range between 0 and 255. Second, a local adaptive binarization processing method is used to separate the background information and the speckle. In the example described herein, the local adaptive binarization includes determining a binarization threshold at a pixel location based on a pixel value distribution of a domain block of pixels. In this way, it does not matter whether the brightness of the entire frame is uneven, because the frame can be configured to be compared and screened, for example, by a threshold applied within a set window size (see equations [1], [2]):

[0083]

[0084]

[0085] in:

[0086] I(x,y) indicates the grayscale value of any pixel in the original image;

[0087] I′(x,y) is the pixel value after local adaptive binarization;

[0088] T(x,y) is a locally adaptive binarization threshold, which can be obtained by the average value of multiple pixels in a window with the grayscale pixel value I(x,y) as the center and r as the radius.

[0089] As indicated by the above equation, the binarization result of each pixel is calculated in a window centered on the grayscale pixel value. Compared with global binarization, the adopted local binarization can find the difference of the window even if the window is too dark or too bright in the whole image, so that the speckle characteristics in the too bright or too dark area can still be retained. Therefore, if some areas are considered too bright or too dark under the influence of the background, the speckle of the area may still be different from the surrounding environment pixels in the local range. This process facilitates the extraction of the complete binarized image of the speckle characteristics.

[0090] Since each individual speckle is formed by connecting several pixels, the binarized speckle image will inevitably retain some brighter and smaller speckle-free areas, which are regarded as environmental noise. Therefore, according to some examples and in order to obtain clean and clear speckles, a morphological opening operation is performed, which is a technique known in general image processing but has not been adopted in the deep decoding task until the examples described in this article. The known definition of morphological opening of an image is erosion followed by dilation, and both operations use the same structural element. In some embodiments, dilation and erosion can be combined to remove / filter out these small objects from the image. Environmental noise and smooth the boundaries of large objects. Therefore, in some examples and according to such erosion and dilation operations, the examples described in this article set a reasonable kernel size to substantially retain the speckles (e.g., without significantly changing the size of the speckles) and remove noise.

[0091] Reference now Figure 5 , an example of a selected image region 500 is shown according to some examples described herein, wherein an unprocessed image is shown at 510 and a pre-processed image after local adaptive binarization and morphological opening operation to remove small noise data of the binarized image is shown at 520. The currently known techniques do not apply any concept of local adaptive binarization and morphological opening operation to remove small noise data of the binarized image. In some examples, the morphological opening of the image layer of the speckle pre-processing operation includes first corroding and then dilating the binarized speckle image to filter out these ambient noises without significantly changing the size of the speckle speckles. In the known existing deep decoding algorithms, this additional layer of speckle pre-processing is not considered. However, the inventors have recognized that a clearer speckle image and a more complete data set are very important for the accuracy of subsequent deep decoding.

[0092] In this way, the corresponding infrared speckle image and the relevant matching area on the reference speckle pre-processed image can be cropped / extracted. In addition, in this example, there is no need to process non-target areas, nor is there a need to process each frame when no target is detected. In some examples, in order to help the deep decoding algorithm, focusing on the selected area to save processing time and memory, focusing on the selected area can significantly reduce power consumption and save computing resources.

[0093] Return to reference Figure 4 At 450, for example, an efficient search strategy and a lookup table are used to perform speckle matching and optimization acceleration. In the examples described herein, optimization acceleration refers to using a LUT to perform similarity calculations on speckle features, and using an efficient search strategy that adds conditional judgments (see, for example, Figure 8 822, 826). Once the features are extracted before speckle matching, it is only necessary to find the reference speckle features with the highest similarity in the target speckle image. At 460, in this example, sub-pixel level processing is performed, for example, to improve the depth estimation / decoding accuracy of distant targets. Thereafter, target depth information may be generated at 470. The target depth information generated at 470 is sometimes referred to as a depth disparity map, which may be obtained by calculating the row-wise translation relationship between the object speckle image and the reference speckle image. Thereafter, as is known, the actual distance value ('depth information') may be derived by triangulation principles.

[0094] Specifically, in this way and compared to known methods, the combination of binary coding compression with efficient speckle matching and lookup table search methods provides an efficient and memory-efficient algorithm that runs on processors (e.g., Figure 3 The processor 306 in the embodiment of the present invention is used to execute the image processing on the processor 306 in the embodiment of the present invention to obtain accurate target depth information for 3D related applications. In some examples, the binary coding compression can be configured to store, for example, 8 pixels in 1 byte without losing the information of the speckle matching. At the same time, the use of the lookup table search method can be configured to reduce the computational complexity of the matching similarity, especially when the information of 8 pixels is stored in 1 byte. Otherwise, the processor can be configured to divide the 8-bit data into individual bits in order to calculate the matching similarity.

[0095] In this way, the examples described herein are able to improve the performance of speckle image processing without significantly compressing memory usage, without incorporating complex calculations and avoiding significant sacrifice of speckle image resolution, as identified in known methods.

[0096] In addition, some examples described herein may also use the characteristics of binary encoding to represent the pixel values ​​of the binarized speckle image obtained above with '0' or '1', and encode the values ​​of 8 pixels into 8-bit integer values. In this way, in some examples, an image with an original size of height × width bytes only requires a storage space of height × width / 8 (bytes). At the same time, the subsequent matching similarity calculation may also be processed in parallel for 8 pixels to help improve computational efficiency. It should be noted that the example binary encoding method is different from the methods and goals of the known technology. For example, after the local adaptive binarization process (different from the known technology), some examples described herein are able to use compression and encoding methods to store 8 bits in a byte-limited manner. Specifically, the goal of the known technology is to represent a single scattered speckle with a single pixel by extracting the center of the scattered speckle. Generally speaking, each scattered speckle is composed of '5' or '6' or more pixels. Then, the known technology uses a convolution process to compress the data storage space. Therefore, the operation of putting '8' pixels into one byte is not suitable for the compression and encoding methods of the known technology. In addition, some known technologies compress the data storage space, but at the same time sacrifice the resolution of the speckle image, so that the matching accuracy cannot reach the pixel level.

[0097] Although the examples described herein propose to use 8-bit encoding compression, because the 8-bit encoding compression not only compresses the space without losing precision, but also provides conditions for the acceleration of subsequent speckle matching without losing precision, it is conceivable that other bit encoding compression ratios may be used, although they may be less efficient, such as 4 or 16 bits. For example, 4-bit compression is less efficient than 8-bit compression, and 4-bit compression only compresses four times the memory. In addition, 16-bit compression may not be an ideal storage space compression solution, for example, resulting in low efficiency of the lookup table, and / or the LUT also occupies more memory space.

[0098] like Figure 5 As shown, after the above processing, the original infrared speckle image 510 has been successfully converted into a clean and clear speckle binarized image 520, and the memory size is reduced by 8 times. For the reference speckle image, only the preprocessing operation needs to be performed in its initialization stage, and then the reference speckle image is replaced with the preprocessed reference speckle image. Thereafter, for the object speckle image, only the specified target area needs to be processed to derive the corresponding depth information. Since the reference speckle frame is known from the beginning and does not change, it can be processed first. Thereafter, if a target is detected, according to the example described in this article, the process only needs to be cropped from the preprocessed reference speckle frame to obtain the target-related reference speckle image after preprocessing. The object speckle image always changes, so the target speckle needs to be cropped and the speckle preprocessing operation needs to be performed each time.

[0099] Speckle matching and optimization acceleration

[0100] Reference now Figure 6 , shows an example schematic diagram of a speckle matching process 600 according to some examples. An existing process 610 using a pre-processed object speckle image 620 with a target speckle image 622 and a pre-processed reference speckle image 630 with a target template 632 is improved. Once the features are extracted before speckle matching, it is only necessary to find the reference speckle features with the highest similarity in the target speckle image. In this example, an efficient search strategy is adopted to reduce the matching cost, wherein the search strategy utilizes a lookup table (LUT) method to accelerate the 'similarity' calculation of speckle matching. Therefore, in this way, the target speckle image and the pre-processed reference speckle image can be matched quickly and accurately. Thereafter, the depth disparity information of the target speckle image can be derived.

[0101] As is well known, it is currently difficult to find a matching pixel for each single pixel. Therefore, in order to find a matching pixel for each single pixel, in the exemplary schematic diagram of the speckle matching process 600, an N×N pixel speckle descriptor block 654 is selected on the pre-processed target speckle image 652 and centered at each pixel P(x,y) as the 'target speckle descriptor' 642 to be matched. Similarly, an N×N pixel target template block 662 is selected on the pre-processed reference speckle image of the target area with the same pixel as the center as the 'reference matching speckle descriptor' 664.

[0102] Compared with the target speckle image 652, the speckle spots in the target speckle descriptor 642 will shift left or right as the measured distance changes. Therefore, in some examples, a left and right search range of 'M' is set, which can be obtained from the distance measurement range. Thereafter, in some examples, the reference matching speckle descriptor 664 will slide a total of 'M' pixels to the left and right in the row direction. Therefore, for one target speckle descriptor 642, there will be 'M+1' reference matching speckle blocks.

[0103]

[0104] Wherein, in the calculation formula of equation [3]:

[0105] M is the length of the search range of the target speckle descriptor 642;

[0106] left_slide_num 650 is the slide number of the target speckle descriptor 642 to search to the left on the reference speckle image; and

[0107] right_slide_num 656 is the slide number of the target speckle descriptor 642 searched to the right on the reference speckle image.

[0108] In order to improve the efficiency of the matching search, some examples describe an improved and optimized search strategy. A large number of tests performed and statistical analysis of these tests show that the best matching pixels of two adjacent pixels in the target speckle image are often very close and can be found within the search range of M / 6. Therefore, in some examples, a threshold can be set to measure the reliability of speckle matching. In the row direction of such a scene, in one example, when the matching rate of the best matching point is greater than the threshold, it can be assumed that the next target speckle descriptor to be matched may not require M+1 matches, but may only need the position of the best matching point of the previous target speckle descriptor to be matched as the starting point. Therefore, in this example, sliding left and right only requires a range of M / 6 pixels, so the number of matching pixels is M / 6+1.

[0109] In this way, the matching cost can be reduced by about six times, thereby significantly improving the matching efficiency.

[0110]

[0111] Where, in equation [4]:

[0112] T is the threshold for determining whether the best matching result is qualified; and

[0113] max matching_rate is the maximum matching rate of speckle.

[0114] Next, according to the evaluation criteria that can be used, for example, depending on the corresponding application, the best matching point of the target speckle image can be found using the matching between the target speckle descriptors 642, 654 and their multiple reference matching speckle descriptors. In some examples, the Hamming distance can be selected as the similarity criterion between two speckle descriptors. In some examples, the Hamming distance can be selected because it can utilize binary coded data and can be calculated using the LUT method, thereby greatly improving the calculation efficiency. Those skilled in the art will easily understand and appreciate that other similarity criteria such as cosine distance can be used for other applications and embodiments.

[0115] Specifically, in this example, the number of XOR values ​​of the two speckle descriptors is calculated to represent the number of mismatched pixels. According to the result of the Hamming distance, the similarity of the two speckle descriptors can be evaluated by the matching rate, as shown in equation [5], that is:

[0116]

[0117] Where, in equation [5]:

[0118] H indicates the Hamming distance; and

[0119] s1 and s2 are the number of pixel values ​​of '1' in the target speckle descriptor and the reference matching speckle descriptor, respectively.

[0120] As understood and appreciated by those skilled in the art, generally, the higher the matching rate, the higher the confidence level of the matching speckle descriptor.

[0121] In addition, in order to speed up the calculation of matching similarity, the lookup table (LUT) method described in some examples above can be introduced to efficiently calculate the Hamming distance. In this example, LUT is used to speed up the Hamming distance calculation, mainly based on the use of binary coding compression in speckle feature extraction. It should be noted that there is currently no method that uses binary coding compression. Although the example described in this article binary encodes the speckle image and simplifies it to only '0' and '1', the calculation amount of the Hamming distance is still not small. However, it may be shown that determining the Hamming distance requires a lot of calculations to count how many '1's there are in the result. For example, each loop to calculate the matching similarity requires N×N accumulations. It is well known that an 8-bit binary code can describe a maximum of 256 values, and the number of bits that are '1' in each value cannot be directly derived. Therefore, in some examples, a 256-element LUT is introduced to represent all results of the number of bits that are 1 for the 256 values, such as Figure 7 As shown, Figure 7 An example of a table 700 that can be used to represent the number of bits that are '1' in an 8-bit binary value according to some examples is shown. In this way, the number of cyclic accumulations is reduced from N×N to (N×N) / 8, making the calculation of matching similarity eight times faster than without the introduction of the LUT method.

[0122] Finally, the lateral difference between the central pixel of the best matching reference speckle descriptor and the central pixel of the target speckle descriptor is the disparity value of the target pixel to be matched. For a 3D structured light speckle camera / system, it is known that the speckle pattern moves at different distances, and the size and distribution pattern of the speckle do not change with distance. Therefore, in some examples, the triangulation principle is used to calculate the disparity value to derive the distance value.

[0123] Reference now Figure 8, shows an example of a high-level flow chart of a speckle matching and search process 800 according to some examples. To find a matching pixel for each individual pixel, each pixel P(x,y) is defined as a corresponding point on the original pre-processed target speckle image at 802. At 804, it is determined whether 'y' is less than a target height 'h'. If at 804 'y' is not less than the target height 'h', the flow chart loops to 828 and outputs a target depth map. If 'y' is less than the target height 'h', then at 806 it is determined whether 'x' is less than a target width 'w'. If at 806 'x' is not less than the target width 'x', then at 808 'y' is incremented and the flow chart loops back to 804.

[0124] If 'x' is less than the target width 'x' at 806, then an N×N pixel block of the target speckle descriptor for each pixel P(x,y) is selected at 810. It is then determined whether the search range is less than 'M' at 812. If the search range is not less than 'M' at 812, the process loops to 820. If the search range is less than 'M' at 812, then an N×N pixel block of the reference matching speckle descriptor for each pixel P(x,y) is selected at 814. At 816, the matching similarity of the target speckle descriptor and the reference matching speckle descriptor for the corresponding pixel P(x,y) is calculated. The search range is then incremented at 818, and the process loops back to 812.

[0125] The process loops until the search range is not less than 'M' at 812, and the best matching point of the corresponding pixel P(x,y) with the maximum similarity is obtained at 820. At 822, it is determined whether the maximum similarity is greater than a threshold. If at 822, the maximum similarity is not greater than the threshold, the flowchart moves to 824, where 'x' is incremented (as it moves along the row) and the process loops back to 806. Alternatively, if at 822 the maximum similarity is greater than the threshold, then at 826 M=M / 6, and the flowchart moves to 824, where 'x' is incremented (as it moves along the row) and the process loops back to 806. Therefore, in this example, only a range of M / 6 pixels is required (i.e., for left and right sliding), so the number of matching pixels is M / 6+1. In this way, the matching cost can be reduced by about six times, thereby significantly improving the matching efficiency, as shown in equation [4].

[0126] Improving long-distance depth decoding accuracy using sub-pixel processing

[0127] In some examples, a method of using sub-pixel level (SPL) processing to improve the accuracy of long-distance depth decoding can be used. In one example, when the target is far away from the camera lens, this SPL processing method can be adaptive and used. SPL operation will increase computing resources, but the inventors have recognized and understood that not every frame needs this operation. Therefore, in some examples, a threshold can be added to facilitate the adaptive use of SPL. In some examples, the threshold can be the size of the target image, which can be obtained after multiple tests and analysis of statistical data. In one example, when the target size is less than the threshold, SPL can be adaptively used to enhance the depth information restoration effect. The SPL processing method is preferably implemented before the speckle feature extraction preprocessing and binary encoding compression operation. In this case, there are usually only a small number of speckles in the area of ​​interest, so the depth change will not be obvious. However, the current speckle matching results are pixel-level, which means that if the depth difference of the distant object is less than one pixel unit, it is considered that the depth has not changed. This is because the depth decoding algorithm is of course related to the distance. For example, without this feature, the depth decoding algorithm may only be able to restore a satisfactory depth map at a distance of, for example, 25-60cm. However, when adopting this SPL processing feature, the depth estimation / decoding algorithm can support targets at farther distances of 60-90cm with the same depth map accuracy.

[0128] In order to improve the depth resolution of the target far away from the lens, the examples described in this article provide a SPL processing method, in which the reference speckle image and the target speckle image before preprocessing are interpolated. In some examples, the reference speckle image and the target speckle image before preprocessing can be subjected to bilinear interpolation. Mathematically speaking, the bilinear interpolation process is an extension of the linear interpolation process with respect to the interpolation function with two variables. Therefore, the pixel information between two pixels can be refined. In some examples, it can be imagined that the pixel value of the new pixel can be derived from the known pixel values ​​within the range of the 'x' and 'y' directions of the pixel.

[0129] Since some long-distance speckle may be lost, the lost speckle information can be derived through interpolation processing to improve accuracy. In this way, the speckle feature extraction preprocessing and binary coding compression operations as well as the speckle matching and optimization acceleration methods are all based on sub-pixel speckle images. In this way, when speckle matching is performed on distant objects, depth disparity values ​​with sub-pixel accuracy can be obtained; that is, the unit of the depth result is 0.5 pixels. To perform this operation, in some examples, a threshold of the target size can be given to distinguish whether to use sub-pixel processing, and the threshold is defined according to actual application scenarios and requirements. In this example, the threshold can be set to the size of the target image, which can be obtained after performing multiple tests to determine the distance of the object and the resulting statistical data. For example, when the target size is smaller than the threshold, it can be assumed that SPL is needed to enhance the depth information restoration effect, such as Fig. 9 shown.

[0130] Reference now Fig. 9 , shows an example of an image 900 before and after sub-pixel level processing 940 according to some examples. Here, the image shows: a pre-processed target template 910; a pre-processed target template 912 after SPL optimization; a pre-processed target 920; a pre-processed target 922 after SPL optimization; a target depth disparity map 930; and a target depth disparity map 932 after SPL optimization.

[0131] In some examples, it is contemplated that a processor (e.g. Figure 3 The processor 306 in the target speckle image processing unit 306 may perform a local or global adaptive binarization process, which searches for speckle features of a stored reference speckle image in the LUT memory when used for SPL optimization, and matches the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image.

[0132] Actual distance value calculation

[0133] After the above processing, the actual depth value can be calculated based on the speckle projection principle and triangulation principle of 3D structured light. The calculation formula for depth calculation can be expressed as:

[0134]

[0135] in: f is the focal length of the camera;

[0136] H is the distance between the reference plane and the camera;

[0137] L is the distance between the infrared camera and the speckle projector;

[0138] Δx is the depth disparity value of the target, which is obtained through the above operation; and

[0139] D is the distance between the target and the surface of the camera speckle projector.

[0140] In the foregoing specification, examples have been described with reference to specific exemplary embodiments. However, it will be apparent that various modifications and changes may be made to the specific examples without departing from the scope as set forth in the appended claims, and the claims are not limited to the specific examples described above.

[0141] As discussed herein, connection can be suitable for, for example, transmitting a signal from a corresponding node, unit or device via an intermediate device or transmitting a signal to any type of connection to a corresponding node, unit or device. Therefore, unless otherwise implied or stated, connection can be, for example, a direct connection or an indirect connection. Connection can be shown or described as a single connection, multiple connections, a unidirectional connection or a bidirectional connection. However, different embodiments can change the implementation scheme of the connection. For example, a separate unidirectional transmission connection can be used instead of a bidirectional transmission connection, and vice versa. And, multiple connections can be replaced by a single connection that transmits multiple signals in a continuous manner or in a time division multiplexing manner. Similarly, a single connection carrying multiple signals can be divided into various different connections that carry a subset of these signals. Therefore, there are many options for transmitting signals. Those skilled in the art will recognize that the architecture described herein is only exemplary, and in fact, many other architectures that realize the same functionality can be implemented.

[0142] Any arrangement of components that achieve the same functionality is effectively 'associated' so as to achieve the desired functionality. Thus, any two components herein combined to achieve a particular functionality may be considered to be 'associated' with each other so as to achieve the desired functionality, regardless of architectures or intermediate components. Likewise, any two components so associated may also be considered to be 'operably connected' or 'operably coupled' to each other so as to achieve the desired functionality.

[0143] In addition, it will be appreciated by those skilled in the art that the boundaries between the operations described above are merely schematic. Multiple operations may be combined into a single operation, a single operation may be distributed in additional operations, and the execution of the operations may overlap at least partially in time. In addition, alternative embodiments may include multiple instances of specific operations, and the order of operations may be changed in various other embodiments. In addition, for example, in one embodiment, the example shown may be implemented as a circuit system located on a single integrated circuit or in the same device.

[0144] In some examples, various components within the device with a light source for processing depth information circuitry may be implemented as discrete or integrated components, so the final structure is application specific or a matter of design choice. Since the illustrated embodiments can be implemented for the most part using electronic components and circuits known to those skilled in the art, in order to understand and appreciate the basic concepts described herein and in order not to obscure or deviate from the teachings herein, the details will not be explained to any greater extent than deemed necessary as described below. Those skilled in the art will appreciate that in some cases, the degree of integration of the device with a light source for processing depth information circuitry or components may depend on the implementation.

[0145] In addition, for example, the example or part thereof may be implemented as a software or code representation of a physical circuit system or may be converted into a software or code representation of a logical representation of a physical circuit system, for example, in the form of a hardware description language of any appropriate type. In addition, the examples described herein are not limited to physical devices or units implemented in non-programmable hardware, but may also be applied to programmable devices or units that can perform desired sampling errors and compensation by operating according to suitable program codes, such as microcomputers, personal computers, notebooks, personal digital assistants, electronic games, automobiles and other embedded systems, cellular phones and various other wireless devices, which are collectively referred to as 'computer systems' in this application. However, other modifications, variations and alternatives are also possible. Therefore, the specification and drawings should be viewed in an illustrative rather than a restrictive sense.

[0146] In the claims, any reference numerals placed between parentheses should not be interpreted as limiting the claims. The word 'comprising' does not exclude the presence of other elements or steps other than those listed in the claims. In addition, as used herein, the term 'one' or 'an' is defined as one or more than one. Similarly, the use of introductory phrases such as 'at least one' and 'one or more' in the claims should not be interpreted as implying that any particular claim containing such introduced claim elements introduced by the indefinite article 'one' or 'an' is limited to an invention containing only one such element, even when the same claim includes the introductory phrase 'one or more' or 'at least one' and an indefinite article such as 'one' or 'an'. The same applies to the use of definite articles. Unless otherwise stated, terms such as 'first' and 'second' are used to arbitrarily distinguish the elements described by such terms. Therefore, these terms are not necessarily intended to indicate the priority or other priority of such elements in time. The mere fact that certain measures are described in claims that are different from each other does not indicate that the combination of these measures cannot be used advantageously.

Claims

1. A three-dimensional 3D structured light camera device (310), characterized in that: include: an infrared (IR) light source (304) configured to illuminate a three-dimensional 3D target object, a speckle emitter (302), and an IR camera (308); a lookup table LUT memory (307) configured to store a reference speckle image of the 3D target object; as well as a processor (306) operably coupled to the light source and the LUT memory (307) and configured to: performing object detection on the target speckle image; Identifying a 3D target object focus area of ​​the target speckle image; performing a local adaptive binarization process, the local adaptive binarization process searching for speckle features of a stored reference speckle image in the LUT memory and matching the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image; searching for matching speckle features of the target speckle image; and Depth value calculation of the 3D target object is performed in response to the target speckle image.

2. The 3D structured light camera device (310) according to claim 1, characterized in that: In response to the processor being configured to find the matching speckle feature of the target speckle image, the processor is further configured to: performing a morphological operation on the matching speckle features of the stored reference speckle image and the target speckle image, the morphological operation producing a noise-reduced target speckle image; performing binary coding compression on the stored reference speckle image and the noise-reduced target speckle image; and Depth value calculation of the 3D target object is performed in response to performing the binary coding compression on the noise-reduced target speckle image.

3. The 3D structured light camera device (310) according to claim 2, characterized in that: The processor being configured to perform binary encoding compression on the noise-reduced target speckle image includes the processor being configured to compress an eight-pixel image into one-byte image data.

4. The 3D structured light camera device (310) according to claim 3, characterized in that: The LUT memory (307) is configured to store at least one byte of image data of eight pixel speckle features of the stored reference speckle image.

5. The 3D structured light camera device (310) according to any one of the preceding claims, characterized in that The processor (306) is configured to match the one-byte speckle feature of the target speckle image with the searched speckle feature of the stored reference speckle image.

6. The 3D structured light camera device (310) according to any of the preceding claims, characterized in that The processor (306) is configured to perform object detection on the target speckle image, including: the processor (306) is configured to obtain one or more coordinate parameters of the target bounding box, and the processor (306) is configured to adaptively extract the 3D target object focus area from the one or more coordinate parameters for processing.

7. The 3D structured light camera device (310) according to any of the preceding claims, characterized in that The processor (306) is configured to convert the target speckle image and the reference speckle image from original red-green-blue RGB speckle images into single-channel grayscale speckle images with pixel values ​​ranging from '0' to '255'.

8. The 3D structured light camera device (310) according to any one of the preceding claims, characterized in that The processor (306) being configured to perform local adaptive binarization includes the processor being configured to separate background information and speckle of the target speckle image.

9. An integrated circuit for a 3D structured light camera device (310), characterized in that: The integrated circuit comprises a processor according to any preceding claim.

10. A method for performing depth value calculation in a three-dimensional 3D structured light camera device (310), characterized in that: The method comprises: Illuminating a three-dimensional 3D target object; storing a reference speckle image of the 3D target object; performing object detection on the target speckle image of the 3D target object; Identifying a 3D target object focus area of ​​the target speckle image; performing a local adaptive binarization process, the local adaptive binarization process comprising searching for speckle features of a stored reference speckle image in a lookup table (LUT) memory and matching the speckle features of the target speckle image with the searched speckle features of the stored reference speckle image; searching for matching speckle features of the target speckle image; and Depth value calculation of the 3D target object is performed in response to the target speckle image.

Images