Method, device, medium and equipment for determining effective depth value of ITOF module

Abstract

This invention provides a method, apparatus, medium, and device for determining the effective depth value of an ITOF module. The method involves determining the corresponding target image region in the ITOF image based on the coordinates of each spot point; parsing the target image region to obtain an initial effective depth value; and filtering the initial effective depth value to obtain a target effective depth value. Thus, by first determining the corresponding target image region in the ITOF image based on the coordinates of the spot points, the effective depth value can be determined simply by parsing the target image region, without needing to calculate the depth value of all pixels. This significantly reduces the computational burden and improves the efficiency of obtaining the effective depth value.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, medium and device for determining the effective depth value of an ITOF module. Background Technology

[0002] Currently, Time of Flight (TOF) cameras, as a type of 3D depth-sensing camera, are widely used in applications such as the measurement and recognition of 3D objects.

[0003] With the increasing number of application scenarios and the growing demands of end-user applications, spot-ITOF technology has emerged. Spot-ITOF is a lidar technology that concentrates laser energy at a characteristic point (spot point). Spot-ITOF modules, as an emerging ITOF technology, have received widespread attention in recent years.

[0004] Currently, depth data calculation requires first parsing the RAW image of the spot-ITOF module and calculating the depth values ​​of all pixels before extracting the effective depth values ​​of the spot points. However, calculating the depth values ​​of all pixels is time-consuming and computationally burdensome, leading to a decrease in the efficiency of acquiring effective depth data. Summary of the Invention

[0005] To address the problems existing in the prior art, embodiments of the present invention provide a method, apparatus, medium, and device for determining the effective depth value of an ITOF module, so as to solve or partially solve the technical problems of long time consumption and increased computational burden in determining the effective depth data of spot-ITOF modules in the prior art.

[0006] A first aspect of the present invention provides a method for determining the effective depth value of an ITOF module, the method comprising:

[0007] The corresponding target image region is determined in the ITOF image based on the coordinates of each spot point; the spot point is the speckle light spot of the ITOF module.

[0008] The target image region is analyzed to obtain an initial effective depth value;

[0009] The initial effective depth values ​​are filtered to obtain the target effective depth values.

[0010] In the above scheme, before determining the corresponding target image region in the ITOF image based on the coordinates of each spot point, the method further includes:

[0011] Obtain the model number of the ITOF module;

[0012] The coordinates of each spot point are found in a preset mapping table according to the model of the ITOF module; wherein the mapping table stores the correspondence between the model of the ITOF module and the coordinates of each spot point.

[0013] In the above scheme, determining the corresponding target image region in the ITOF image based on the coordinates of each spot point includes:

[0014] For any given spot point, the target image region is expanded outward from the spot point's coordinates to form a target image region of a preset size.

[0015] In the above scheme, the step of parsing the target image region to obtain an initial effective depth value includes:

[0016] Obtain the depth value of each pixel in the target image region;

[0017] The depth values ​​of each pixel are corrected to obtain the initial effective depth value of each pixel.

[0018] In the above scheme, the step of filtering the initial effective depth value to obtain the target effective depth value includes:

[0019] For any target image region, obtain the confidence value for each initial valid depth value;

[0020] The initial effective depth corresponding to the maximum confidence value is determined as the target effective depth value.

[0021] In the above scheme, after filtering the initial effective depth value to obtain the target effective depth value, the method further includes:

[0022] Save the effective depth values ​​of all target region images to the depth map to form the target depth map to be output.

[0023] A second aspect of the present invention provides an apparatus for determining the effective depth value of an ITOF module, the apparatus comprising:

[0024] The determining unit is used to determine the corresponding target image region in the ITOF image based on the coordinates of each spot point; the spot point is the speckle light spot of the ITOF module.

[0025] The parsing unit is used to parse the target image region to obtain an initial effective depth value;

[0026] The filtering unit is used to filter the initial effective depth value to obtain the target effective depth value.

[0027] In the above scheme, the filtering unit is specifically used for:

[0028] For any target image region, obtain the confidence level of each initial valid depth value;

[0029] The initial effective depth corresponding to the maximum confidence level is determined as the target effective depth value.

[0030] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the first aspects.

[0031] A fourth aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method described in any of the first aspects.

[0032] This invention provides a method, apparatus, medium, and device for determining the effective depth value of an ITOF module. The method involves determining a corresponding target image region in an ITOF image based on the coordinates of each spot point; the spot point is the speckle light point of the ITOF module; the target image region is analyzed to obtain an initial effective depth value; the initial effective depth value is then filtered to obtain a target effective depth value. Thus, by first determining the corresponding target image region in the ITOF image based on the coordinates of the spot points, the effective depth value can be determined simply by analyzing the target image region, without needing to calculate the depth value of all pixels. This significantly reduces the computational burden and improves the efficiency of obtaining the effective depth value. Attached Figure Description

[0033] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0034] In the attached diagram:

[0035] Figure 1 A flowchart illustrating a method for determining the effective depth value of an ITOF module according to an embodiment of the present invention is shown.

[0036] Figure 2 A schematic diagram of a device for determining the effective depth value of an ITOF module according to an embodiment of the present invention is shown.

[0037] Figure 3 A schematic diagram of a computer device structure according to an embodiment of the present invention is shown;

[0038] Figure 4A schematic diagram of a computer-readable storage medium structure according to an embodiment of the present invention is shown. Detailed Implementation

[0039] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0040] This embodiment provides a method for determining the effective depth value of an ITOF module, such as... Figure 1 As shown, the method includes the following steps:

[0041] S110, determine the corresponding target image region in the ITOF image based on the coordinates of each spot point; the spot point is the speckle light spot of the ITOF module.

[0042] To better understand the technical solution of this application, we will first introduce the spot-ITOF module.

[0043] Based on the different light fields formed by the emitted light from the ITOF transmitter, ITOF modules can be divided into flood-ITOF and spot-ITOF. The flood-ITOF module exhibits a relatively uniform distribution of emitted light energy across the entire field of view. The spot-ITOF module decomposes the emitted energy, concentrating it at the spot point, allowing for measurement distances exceeding 5 meters and achieving high accuracy with very low power consumption.

[0044] The number of spot points includes multiple points, and the number and location of spot points are mainly determined by the model of the spot-ITOF module.

[0045] In one implementation, before determining the corresponding target image region in the ITOF image based on the coordinates of each spot point, the method further includes:

[0046] Obtain the model number of the ITOF module;

[0047] The coordinates of each spot point are found in a preset mapping table based on the model of the ITOF module. The mapping table stores the correspondence between the model of the ITOF module and the coordinates of each spot point.

[0048] In one implementation, determining the corresponding target image region in the ITOF image based on the coordinates of each spot point includes:

[0049] For any given spot point, expand outwards from the spot point's coordinates to form a target image region of a preset size.

[0050] For example, based on the coordinates of the spot point, expand outwards by 4 pixels in each direction to form a 4x4 pixel target image area.

[0051] This embodiment takes into account that since the effective depth values ​​are mainly concentrated in the spot point area, the corresponding target image area can be determined in the ITOF image based on the coordinates of each spot point. Subsequently, the target image area can be directly parsed, and the non-target image area does not need to participate in the parsing process, reducing the subsequent computational burden and improving the efficiency of determining the effective depth value.

[0052] S111, parse the target image region to obtain an initial effective depth value;

[0053] In one implementation, the target image region is parsed to obtain an initial effective depth value, including:

[0054] Obtain the depth value of each pixel in the target image region;

[0055] The depth values ​​of each pixel are corrected to obtain the initial effective depth value for each pixel.

[0056] Specifically, for any target image region, an image parsing function can be used to parse the target image region and obtain the depth value of each pixel. Other parsing methods can also be used to parse the target image region, which will not be detailed here.

[0057] For current ITOF modules, when the infrared light emitted by the ITOF module's transmitter becomes weaker after reflection, the signal amplitude received by the ITOF module's receiver also weakens, resulting in a decrease in the signal-to-noise ratio. In this case, false edges may appear in the depth value, and the depth value is easily interfered with. Therefore, it is necessary to correct the depth value of each pixel to obtain the initial effective depth value for each pixel.

[0058] For example, an adaptive image segmentation method can be used to segment the target image region into multiple pixel blocks; then, an improved structure tensor can be used as an edge detector to detect invalid and flying pixels in the target image region; finally, bicubic interpolation can be used to correct the depth value of flying pixels, and invalid pixels can be directly removed.

[0059] S112, the initial effective depth value is filtered to obtain the target effective depth value.

[0060] While most invalid and flying pixels can be identified, erroneous pixels may appear in gaps when objects are close together, affecting the validity of the depth values. Therefore, it is necessary to filter the initial valid depth values ​​to obtain the target valid depth values.

[0061] In one implementation, filtering the initial effective depth values ​​to obtain the target effective depth value includes:

[0062] For any target image region, obtain the confidence value for each initial valid depth value;

[0063] The initial effective depth corresponding to the maximum confidence value is determined as the target effective depth value.

[0064] When analyzing the target image, both depth data and confidence data can be obtained simultaneously. Therefore, the confidence value of the initial effective depth value of each pixel can be determined based on the confidence data. The higher the confidence value, the more accurate the initial effective depth value.

[0065] For any target image region, the initial effective depth value with the highest confidence value can be determined as the target effective depth value. Ultimately, multiple target effective depth values ​​can be obtained.

[0066] In this way, the corresponding target image region is first determined in the ITOF image based on the coordinates of the spot point. Only the target image region needs to be analyzed to determine the effective depth value of the target. There is no need to analyze the entire ITOF image or calculate the depth value of all pixels. Therefore, the computational burden can be greatly reduced and the efficiency of obtaining the effective depth value can be improved.

[0067] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a device for determining the effective depth value of an ITOF module, such as... Figure 2 As shown, the device includes:

[0068] Determining unit 21 is used to determine the corresponding target image region in the ITOF image based on the coordinates of each spot point; the spot point is the speckle light spot of the ITOF module;

[0069] The parsing unit 22 is used to parse the target image region to obtain an initial effective depth value;

[0070] The filtering unit 23 is used to filter the initial effective depth value to obtain the target effective depth value.

[0071] In one implementation, the filtering unit 23 is specifically used for:

[0072] For any target image region, obtain the confidence level of each initial valid depth value;

[0073] The initial effective depth corresponding to the maximum confidence level is determined as the target effective depth value.

[0074] Since the apparatus described in the embodiments of this invention is used for implementing the method of determining the effective depth value of an ITOF module according to the embodiments of this invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in the embodiments of this invention, and therefore will not be described in detail here. All apparatuses used in the methods of the embodiments of this invention fall within the scope of protection of this invention.

[0075] Based on the same inventive concept, this embodiment provides a computer device 300, such as... Figure 3 As shown, the system includes a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it performs the following steps:

[0076] Determine the corresponding target image region in the ITOF image based on the coordinates of each spot point;

[0077] The target image region is analyzed to obtain an initial effective depth value;

[0078] The initial effective depth values ​​are filtered to obtain the target effective depth values.

[0079] Based on the same inventive concept, this embodiment provides a computer-readable storage medium 400, such as... Figure 4 As shown, a computer program 411 is stored thereon, which, when executed by a processor, performs the following steps:

[0080] Determine the corresponding target image region in the ITOF image based on the coordinates of each spot point;

[0081] The target image region is analyzed to obtain an initial effective depth value;

[0082] The initial effective depth values ​​are filtered to obtain the target effective depth values.

[0083] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:

[0084] This invention provides a method, apparatus, medium, and device for determining the effective depth value of an ITOF module. The method involves determining a corresponding target image region in an ITOF image based on the coordinates of each spot point; the spot point is the speckle light point of the ITOF module; the target image region is analyzed to obtain an initial effective depth value; the initial effective depth value is then filtered to obtain a target effective depth value. Thus, by first determining the corresponding target image region in the ITOF image based on the coordinates of the spot points, the effective depth value can be determined simply by analyzing the target image region, without needing to calculate the depth value of all pixels. This significantly reduces the computational burden and improves the efficiency of obtaining the effective depth value.

[0085] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0086] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0087] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0088] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0089] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0090] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components of the gateway, proxy server, or system according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0091] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0092] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining the effective depth value of an ITOF module, characterized in that, The method includes: The corresponding target image region is determined in the ITOF image based on the coordinates of each spot point; the spot point is the speckle light spot of the ITOF module. The target image region is analyzed to obtain an initial effective depth value; The initial effective depth values ​​are filtered to obtain the target effective depth values; wherein, Before determining the corresponding target image region in the ITOF image based on the coordinates of each spot point, the method further includes: Obtain the model number of the ITOF module; The coordinates of each spot point are found in a preset mapping table according to the model of the ITOF module; wherein the mapping table stores the correspondence between the model of the ITOF module and the coordinates of each spot point in advance. The step of determining the corresponding target image region in the ITOF image based on the coordinates of each spot point includes: For any spot point, expand outward from the spot point's coordinates to form a target image region of a preset size. The step of filtering the initial effective depth values ​​to obtain the target effective depth values ​​includes: For any target image region, obtain the confidence value for each initial valid depth value; The initial effective depth corresponding to the maximum confidence value is determined as the target effective depth value.

2. The method as described in claim 1, characterized in that, The step of parsing the target image region to obtain an initial effective depth value includes: Obtain the depth value of each pixel in the target image region; The depth values ​​of each pixel are corrected to obtain the initial effective depth value of each pixel.

3. The method as described in claim 1, characterized in that, After filtering the initial effective depth values ​​to obtain the target effective depth values, the method further includes: Save the effective depth values ​​of all target region images to the depth map to form the target depth map to be output.

4. A device for determining the effective depth value of an ITOF module, characterized in that, The device includes: The determining unit is used to determine the corresponding target image region in the ITOF image based on the coordinates of each spot point; the spot point is the speckle light spot of the ITOF module. The parsing unit is used to parse the target image region to obtain an initial effective depth value; A filtering unit is used to filter the initial effective depth values ​​to obtain target effective depth values; wherein, Before determining the corresponding target image region in the ITOF image based on the coordinates of each spot point, the method further includes: Obtain the model number of the ITOF module; The coordinates of each spot point are found in a preset mapping table according to the model of the ITOF module; wherein the mapping table stores the correspondence between the model of the ITOF module and the coordinates of each spot point in advance. The step of determining the corresponding target image region in the ITOF image based on the coordinates of each spot point includes: For any spot point, expand outward from the spot point's coordinates to form a target image region of a preset size. The filtering unit is specifically used for: For any target image region, obtain the confidence level of each initial valid depth value; The initial effective depth corresponding to the maximum confidence level is determined as the target effective depth value.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-4.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1-4.

Images