Method, apparatus, and medium for checking speckle-encoded pattern

Abstract

This disclosure relates to a method, apparatus, electronic device, and medium for verifying speckle coding patterns. The method includes: acquiring a target speckle image corresponding to a speckle coding pattern to be verified; decoding the target speckle image to obtain a decoding result corresponding to each pixel position in the target speckle image; and verifying the validity of the coding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position. This facilitates determining whether the generated speckle coding pattern is valid, thereby improving the accuracy of the determined depth information when subsequently determining depth information based on the target speckle image.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device and medium for verifying speckle coding patterns. Background Technology

[0002] 3D reconstruction technology is a crucial research topic in machine vision, referring to the reconstruction of the 3D spatial geometry of a 3D object from its image. Common 3D reconstruction methods include using triangulation based on the parallax principle of dual cameras to obtain depth information, or using structured light to obtain speckle coding and thus depth information. Coding methods can include temporal coding, spatial coding, and so on. For example, Figure 1 This is a schematic diagram illustrating structured light imaging according to an exemplary embodiment. For example... Figure 1 As shown, in structured light imaging, at the transmitting end, a light source projects a pre-set speckle-coded pattern onto the target object, and at the receiving end (e.g., ... Figure 1 The camera in the camera receives the image reflected back from the target object (called a speckle image). Then, the decoding unit in the camera decodes the speckle image in a specific way to obtain the depth information of the target object. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, electronic device and medium for verifying speckle coding patterns.

[0004] According to a first aspect of the present disclosure, a method for verifying speckle coding patterns is provided, comprising:

[0005] Obtain the target speckle image corresponding to the speckle coding pattern to be verified;

[0006] The target speckle image is decoded to obtain the decoding result corresponding to the position of each pixel in the target speckle image;

[0007] The validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern is verified based on the decoding result corresponding to each pixel position.

[0008] Optionally, the decoding result corresponding to the pixel position includes the similarity of the speckle pattern between the target decoding region and other decoding regions within a preset search range at the pixel position;

[0009] The step of verifying the validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position includes:

[0010] For each pixel location, in the decoding results corresponding to the pixel location, determine the K decoding regions that are most similar to the speckle image of the target decoding region, and verify the validity of the encoding pattern corresponding to the pixel location in the speckle encoding pattern based on the positions of the K decoding regions, where K is an integer greater than or equal to 2.

[0011] Optionally, the step of verifying the validity of the encoding pattern corresponding to the pixel position in the speckle coding pattern based on the positions of the K decoding regions includes:

[0012] Based on the positions of the K decoding regions, determine whether there exist any two decoding regions whose distance is less than or equal to a preset threshold;

[0013] If there are any two decoding regions with a distance less than or equal to a preset threshold, then the encoding pattern corresponding to the pixel position in the speckle coding pattern is determined to be invalid.

[0014] Optionally, determining whether there exist any two decoding regions with a distance less than or equal to a preset threshold based on the positions of the K decoding regions includes:

[0015] In each of the K decoding regions, the coordinates of the center pixel are determined.

[0016] Based on the coordinates of the center pixel of the K decoding regions, determine whether there are any two other decoding regions whose distance is less than or equal to a preset threshold.

[0017] Optionally, obtaining the target speckle image corresponding to the speckle coding pattern to be verified includes:

[0018] Obtain the first speckle coding information when the light source passes through the speckle coding pattern to be verified;

[0019] Based on the first speckle coding information and the preset loss function, the second speckle coding information of the light source projected onto the surface of the target object through the speckle coding pattern to be verified is estimated.

[0020] The target speckle image is constructed based on the estimated second speckle coding information.

[0021] Optionally, the method further includes:

[0022] If there is an invalid target position in the speckle coding pattern, then the coding pattern at the target position is regenerated in the speckle coding pattern to be verified.

[0023] Optionally, the method further includes:

[0024] If there is an invalid target position in the speckle coding pattern, the invalidity degree of the coding pattern corresponding to each position is determined according to the decoding result corresponding to each pixel position in the target speckle image;

[0025] Based on the degree of invalidity, the weight of each pixel at each location in the target speckle image is determined, wherein the weight of the pixel is used to characterize the importance of the pixel when calculating depth information;

[0026] The depth information of the target speckle image is determined based on the weight of each pixel in the target speckle image.

[0027] According to a second aspect of the present disclosure, an apparatus for verifying speckle coding patterns is provided, comprising:

[0028] The acquisition module is configured to acquire the target speckle image corresponding to the speckle coding pattern to be verified;

[0029] The decoding module is configured to decode the target speckle image to obtain the decoding result corresponding to the position of each pixel in the target speckle image;

[0030] The verification module is configured to verify the validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position.

[0031] Optionally, the decoding result corresponding to the pixel position includes the speckle pattern similarity between the target decoding region and other decoding regions within a preset search range at the pixel position; the verification module includes:

[0032] The first determining submodule is configured to, for each pixel position, determine the K most similar decoding regions to the speckle image of the target decoding region from the decoding results corresponding to the pixel position, and verify the validity of the encoding pattern corresponding to the pixel position in the speckle encoding pattern according to the positions of the K decoding regions, where K is an integer greater than or equal to 2.

[0033] Optionally, the first determining submodule includes:

[0034] The second determining submodule is configured to determine, based on the positions of the K decoding regions, whether there exist any two decoding regions whose distance is less than or equal to a preset threshold.

[0035] The third determining submodule is configured to determine that if there are any two decoding regions with a distance less than or equal to a preset threshold, the encoding pattern corresponding to the pixel position in the speckle coding pattern is invalid.

[0036] Optionally, the second determining submodule includes:

[0037] The fourth determining submodule is configured to determine the coordinates of the center pixel in each of the K decoding regions.

[0038] The fifth determining submodule is configured to determine, based on the coordinates of the center pixel of the K decoding regions, whether there are any two other decoding regions whose distance is less than or equal to a preset threshold.

[0039] Optionally, the acquisition module includes:

[0040] The acquisition submodule is configured to acquire the first speckle coding information when the light source passes through the speckle coding pattern to be verified;

[0041] The estimation submodule is configured to estimate, based on the first speckle coding information and a preset loss function, the second speckle coding information projected by the light source onto the surface of the target object through the speckle coding pattern to be verified.

[0042] The construction submodule is configured to construct the target speckle image based on the estimated second speckle coding information.

[0043] Optionally, the device further includes:

[0044] The generation module is configured to regenerate the coding pattern at the target position in the speckle coding pattern to be verified if there is a target position in the speckle coding pattern where the coding pattern is invalid.

[0045] Optionally, the device further includes:

[0046] The first determining module is configured to, if there is a target position in the speckle coding pattern that is invalid, determine the degree of invalidity of the coding pattern corresponding to each position based on the decoding result corresponding to each pixel position in the target speckle image;

[0047] The second determining module is configured to determine the weight of a pixel at each location in the target speckle image based on the degree of invalidity, wherein the weight of the pixel is used to characterize the importance of the pixel when calculating depth information;

[0048] The third determining module is configured to determine the depth information of the target speckle image based on the weight of each pixel in the target speckle image.

[0049] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0050] processor;

[0051] Memory used to store processor-executable instructions;

[0052] The processor is configured as follows:

[0053] Obtain the target speckle image corresponding to the speckle coding pattern to be verified;

[0054] The target speckle image is decoded to obtain the decoding result corresponding to the position of each pixel in the target speckle image;

[0055] The validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern is verified based on the decoding result corresponding to each pixel position.

[0056] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the method for verifying speckle coding patterns provided in the first aspect of the present disclosure.

[0057] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0058] By adopting the above technical solution, after decoding the target speckle image, the validity of the coding pattern corresponding to each pixel position in the speckle coding pattern is verified according to the decoding result corresponding to each pixel position. This makes it easier to know whether the generated speckle coding pattern is valid, and thus improves the accuracy of the determined depth information when determining depth information based on the target speckle image in the future.

[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0061] Figure 1 This is a schematic diagram illustrating a structured light imaging according to an exemplary embodiment.

[0062] Figure 2 This is a flowchart illustrating a method for verifying speckle coding patterns according to an exemplary embodiment.

[0063] Figure 3 This is a block diagram of an apparatus for verifying speckle coding patterns according to an exemplary embodiment.

[0064] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0065] As mentioned in the background art, in related technologies, the validity of the speckle coding pattern in the transmitting end is not verified; the decoding unit directly decodes the speckle image to obtain the depth information of the target object. The inventors have discovered that in structured light imaging, the accuracy of the target object depth information obtained by decoding the speckle image is closely related to the speckle coding pattern. For example, factors affecting the accuracy of depth information include, but are not limited to: the similarity of the coding patterns in different regions of the speckle coding pattern, the ratio of white to black dots in the speckle coding pattern (called the energy efficiency ratio), the sharpness of the speckle coding pattern, and the resolvability. The resolvability refers to the proportion of pixels that can be tolerably lost during decoding; this indicates that pixel loss mostly occurs during the process of the light source projecting onto the target object through the speckle coding pattern. Therefore, the validity of the speckle coding pattern affects the accuracy of the determined target object depth information. Thus, the validity of the speckle coding pattern needs to be verified before determining the target object depth information.

[0066] In view of this, the present disclosure provides a method, apparatus, electronic device and medium for verifying speckle coding patterns, so as to verify the validity of speckle coding patterns through the decoding results of target speckle images, thereby improving the accuracy of the determined depth information.

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0068] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0069] Figure 2 This is a flowchart illustrating a method for verifying speckle coding patterns according to an exemplary embodiment. Figure 2 As shown, the method may include the following steps.

[0070] In step S21, the target speckle image corresponding to the speckle coding pattern to be verified is obtained.

[0071] First, it should be understood that in this disclosure, the target speckle image can be a simulated image or a real pattern captured by a camera.

[0072] In one embodiment, the target speckle image is a simulated image. The target speckle image can be obtained as follows: First, first speckle coding information is obtained when the light source passes through the speckle coding pattern to be verified. Then, based on the first speckle coding information and a preset loss function, second speckle coding information is estimated when the light source, passing through the speckle coding pattern to be verified, projects onto the surface of the target object. The second speckle coding information is obtained by calculating the first coding information using the loss function; that is, it is the estimated speckle coding information of the image formed by the light source projecting onto the target object and reflecting back from the target object. Finally, the target speckle image is simulated based on the estimated second speckle coding information.

[0073] It is worth noting that, in this disclosure, a first model for acquiring the first speckle coding information, a second model for predicting the second speckle coding information, and a third model for constructing the target speckle image can be trained using machine learning. Machine learning is a relatively mature technique in related technologies, and this disclosure does not specifically limit its application.

[0074] For example, in this embodiment, the first encoded information obtained after the light source passes through the speckle coding pattern includes a value of 1 for representing bright areas, a value of 0 for representing dark areas, and other values ​​for representing neither bright nor dark areas, wherein the other values ​​are values ​​between 0 and 1. The loss function is used to represent setting a preset number of other values ​​between 0 and 1 to 0, that is, setting the neither bright nor dark areas in the first encoded information to dark areas. It should be noted that a preset number of other values ​​between 0 and 1 can be randomly set to 0.

[0075] In one embodiment, the target speckle image is a real image captured by a camera. That is, the target speckle image is an image reflected back from the target object after the light source is projected through the speckle-coded pattern to be verified. For example, refer to... Figure 1 The light source passes through the speckle coding pattern to be verified and projects onto the target object, while the camera simultaneously captures an image of the target object to obtain a speckle image. It is worth noting that after passing through the speckle coding pattern, the light source can also project light into a specific space, and the camera can then capture an image of that space to obtain a target speckle image.

[0076] In step S22, the target speckle image is decoded to obtain the decoding result corresponding to the position of each pixel in the target speckle image.

[0077] In this disclosure, the image corresponding to each pixel position in the target speckle image can be decoded according to a preset decoding method to obtain the decoding result corresponding to each pixel position. It is worth noting that the decoding method used in this disclosure is a relatively mature decoding method in related technologies, and this disclosure does not specifically limit it.

[0078] In step S23, the validity of the coding pattern corresponding to each pixel position in the speckle coding pattern is verified based on the decoding result corresponding to each pixel position.

[0079] By adopting the above technical solution, after decoding the target speckle image, the validity of the coding pattern corresponding to each pixel position in the speckle coding pattern is verified according to the decoding result corresponding to each pixel position. This makes it easier to know whether the generated speckle coding pattern is valid, and thus improves the accuracy of the determined depth information when determining depth information based on the target speckle image in the future.

[0080] To facilitate a better understanding of the method for verifying speckle coding patterns provided in this disclosure by those skilled in the art, the method will be described in detail below with reference to a specific embodiment.

[0081] Various decoding methods exist in related technologies. In this disclosure, a decoding method that obtains the similarity of the speckle pattern between the target decoding region and other decoding regions after decoding can be used. In one embodiment, the decoding result corresponding to the pixel position includes the similarity of the speckle pattern between the target decoding region and other decoding regions within a preset search range of the pixel position. For example, assuming the preset search range is 40*40 when decoding the image at each pixel position, if the image at pixel position (100, 100) is decoded, the search region is a rectangular area centered at pixel position (100, 100) with a length and width of 40. During the decoding process, the images within the search region can be decoded sequentially through a decoding window. For example, assuming the decoding step size is 1, there are 1600 decoding regions within the search region, and the region corresponding to the decoding window is the decoding region. The target decoding region can be any one of the 1600 decoding regions. For example, it can be the first decoding region.

[0082] In this embodiment, the specific implementation of step S23, which verifies the validity of the coding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position, may include: for each pixel position, determining the K decoding regions most similar to the speckle image of the target decoding region in the decoding result corresponding to the pixel position, and verifying the validity of the coding pattern corresponding to the pixel position in the speckle coding pattern based on the positions of the K decoding regions, where K is an integer greater than or equal to 2.

[0083] It is worth noting that during the decoding process, each decoding region can be identified to distinguish different decoding regions. For example, assuming that the first decoding region is identified as decoding region 1, the second decoding region is identified as decoding region 2, and so on, within the search area corresponding to the pixel position. Accordingly, in this embodiment, assuming that decoding region 1 is the target decoding region, the decoding result RVector corresponding to the pixel position is as shown in formula (1):

[0084]

[0085] in, This characterizes the similarity of the speckle pattern between decoding region 1 and decoding region 2 within the search area corresponding to the pixel location. This characterizes the similarity of the speckle patterns between decoding region 1 and decoding region 2 within the search area corresponding to the pixel location. The similarity of the speckle pattern between the decoded region 1 and the decoded region S within the search area corresponding to the pixel position is represented by S, where S represents the total number of decoded regions within the search area corresponding to the pixel position.

[0086] After obtaining the decoding result, the K decoding regions that are most similar to the speckle image of the target decoding region are identified. Based on the positions of the K decoding regions, the validity of the coding pattern corresponding to the pixel position in the speckle coding pattern is verified.

[0087] For example, in this decoding result, the similarity is sorted according to numerical value, and the decoding regions corresponding to the top K similarity scores are determined as the K decoding regions most similar to the speckle pattern of the target decoding region. For example, if K is 2, assuming the highest similarity after sorting is... and The speckle patterns in decoding region 1 and decoding region 6 are considered similar. Based on the positions of decoding region 1 and decoding region 6, the validity of the encoding pattern corresponding to the pixel position in the speckle encoding pattern is verified.

[0088] Optionally, verifying the validity of the coding pattern corresponding to the pixel positions in the speckle coding pattern based on the positions of the K decoding regions may include:

[0089] Based on the positions of the K decoding regions, determine whether there exist any two other decoding regions whose distance is less than or equal to a preset threshold;

[0090] If there are any two other decoding regions with a distance less than or equal to a preset threshold, then the encoding pattern corresponding to the pixel position in the speckle coding pattern is determined to be invalid.

[0091] For example, within the K decoding regions, the coordinates of the center pixel of each decoding region are determined, and based on the coordinates of the center pixels of the K decoding regions, it is determined whether there are any two decoding regions whose distance is less than or equal to a preset threshold.

[0092] For example, following the above example, assuming the center pixel coordinates of decoding region 1 and decoding region 6 are (x1, y1) and (x6, y6) respectively, the distance between them is calculated, and it is determined whether the distance is less than or equal to a preset threshold. If it is less than the threshold, it indicates that the distance between the two decoding regions with high speckle pattern similarity is close. In this case, mismatch is very likely to occur when determining the depth information. Therefore, in this embodiment, if the distance between the two decoding regions with high speckle pattern similarity is close, the encoding pattern corresponding to the pixel position in the speckle encoding pattern is determined to be invalid.

[0093] It is worth noting that for each pixel position in the target speckle image, the decoding result corresponding to that pixel position can be obtained. Then, the validity of the coding pattern corresponding to that pixel position in the speckle coding pattern can be verified based on the decoding result. In this way, the validity of the coding image at each position in the speckle coding pattern can be verified, thereby improving the reliability of the speckle coding pattern.

[0094] In one embodiment, after determining that there is an invalid target position in the speckle coding pattern, the speckle coding pattern can be regenerated to ensure that a compliant speckle coding pattern can be obtained. For example, if there is an invalid target position in the speckle coding pattern, the coding pattern at the target position is regenerated in the speckle coding pattern to be verified. It is worth noting that the method of generating the coding pattern is a relatively mature technology in related fields, and this disclosure does not specifically limit it.

[0095] By adopting the above technical solution, if there is an invalid target position with an invalid coding pattern, the coding pattern at the target position is regenerated to obtain a speckle coding pattern that meets the requirements. This can improve the accuracy of the determined depth information when determining depth information based on the target speckle image in the future.

[0096] In another embodiment, if there is a target position with invalid coding pattern in the speckle coding pattern, the invalidity degree of the coding pattern corresponding to each position is determined according to the decoding result corresponding to each pixel position in the target speckle image, and the weight of the pixel corresponding to each position is determined in the target speckle image according to the invalidity degree, wherein the weight of the pixel is used to characterize the importance of the pixel when calculating the depth information, and the depth information of the target speckle image is determined according to the weight of each pixel in the target speckle image.

[0097] In this embodiment, if it is determined that there is an invalid target position in the speckle coding pattern, the invalidity level of the coding pattern corresponding to each position in the speckle coding pattern is determined. For example, the invalidity level of the coding pattern corresponding to a non-target position in the speckle coding pattern is 0, and the invalidity level of the coding pattern corresponding to the target position can be determined based on the decoding results corresponding to the target position and the pixel position.

[0098] For example, suppose that target position a1 in the speckle coding pattern corresponds to pixel position b1 in the target speckle image, and target position a2 in the speckle coding pattern corresponds to pixel position b2 in the target speckle image. For example, if K is 2, and if, according to the above verification method, the distance between the most similar decoding regions of the speckle pattern in the decoding results corresponding to pixel position b1 is T1, and the distance between the most similar decoding regions of the speckle pattern in the decoding results corresponding to pixel position b2 is T2, and T1 is less than T2, then the invalidity of the coding pattern corresponding to target position a1 in the speckle coding pattern is considered greater than the invalidity of the coding pattern corresponding to target position a2.

[0099] For example, if K is greater than 2, the invalidity of the encoded pattern corresponding to the target position can be determined by the number of decoding regions whose distance is less than or equal to a preset threshold among the K decoding regions. If, in the decoding result corresponding to pixel position b1, the number of decoding regions whose distance is less than or equal to the preset threshold among the most similar speckle patterns is 3, and in the decoding result corresponding to pixel position b2, the number of decoding regions whose distance is less than or equal to the preset threshold among the most similar speckle patterns is 2, then the invalidity of the encoded pattern corresponding to target position a1 is considered greater than the invalidity of the encoded pattern corresponding to target position a2. Thus, the invalidity of the encoded pattern corresponding to target position a1 can be determined as 0.8, and the invalidity of the encoded pattern corresponding to target position a2 can be determined as 0.2.

[0100] After determining the invalidity level of the coding pattern corresponding to each position in the speckle coding pattern, the weight of the pixel corresponding to each position in the target speckle image is further determined based on the invalidity level.

[0101] In this disclosure, a pre-defined correspondence between invalidity level and weight can be established, where the invalidity level is inversely proportional to the weight. For example, continuing with the previous example, if the invalidity level of the encoded pattern corresponding to a non-target location is 0, then the weight of the pixel corresponding to the non-target location in the target speckle image is determined to be 1. That is, the weight of all pixels except those at pixel positions b1 and b2 is 1. The weight of the pixel at pixel position b1 is 0.8, and the weight of the pixel at pixel position b2 is 0.2. In this way, the weight of each pixel in the target speckle image can be determined, and the depth information of the target speckle image can then be determined based on the weight of each pixel.

[0102] By adopting the above scheme, after determining the target location in the speckle coding pattern that has an invalid coding pattern, the pixel weight is determined based on the decoding result of the target speckle image, and then the depth information of the target speckle image is determined according to the weight of each pixel. In this way, the accuracy of the determined depth information is improved without the need to regenerate a new coding pattern.

[0103] Based on the same inventive concept, this disclosure also provides an apparatus for verifying speckle coding patterns. Figure 3 This is a block diagram of an apparatus for verifying speckle coding patterns according to an exemplary embodiment. (Refer to...) Figure 3 The apparatus 300 for verifying speckle coding patterns may include:

[0104] The acquisition module 301 is configured to acquire a target speckle image corresponding to the speckle coding pattern to be verified, wherein the target speckle image is an image reflected back from the target object after the light source is projected through the speckle coding pattern to be verified;

[0105] The decoding module 302 is configured to decode the target speckle image to obtain the decoding result corresponding to the position of each pixel in the target speckle image;

[0106] The verification module 303 is configured to verify the validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position.

[0107] Optionally, the decoding result corresponding to the pixel position includes the speckle pattern similarity between the target decoding region and other decoding regions within a preset search range at the pixel position; the verification module 303 includes:

[0108] The first determining submodule is configured to, for each pixel position, determine the K most similar decoding regions to the speckle image of the target decoding region from the decoding results corresponding to the pixel position, and verify the validity of the encoding pattern corresponding to the pixel position in the speckle encoding pattern according to the positions of the K decoding regions, where K is an integer greater than or equal to 2.

[0109] Optionally, the first determining submodule includes:

[0110] The second determining submodule is configured to determine, based on the positions of the K decoding regions, whether there exist any two decoding regions whose distance is less than or equal to a preset threshold.

[0111] The third determining submodule is configured to determine that if there are any two decoding regions with a distance less than or equal to a preset threshold, the encoding pattern corresponding to the pixel position in the speckle coding pattern is invalid.

[0112] Optionally, the second determining submodule includes:

[0113] The fourth determining submodule is configured to determine the coordinates of the center pixel in each of the K decoding regions.

[0114] The fifth determining submodule is configured to determine, based on the coordinates of the center pixel of the K decoding regions, whether there are any two other decoding regions whose distance is less than or equal to a preset threshold.

[0115] Optionally, the acquisition module 301 includes:

[0116] The acquisition submodule is configured to acquire the first speckle coding information when the light source passes through the speckle coding pattern to be verified;

[0117] The estimation submodule is configured to estimate, based on the first speckle coding information and a preset loss function, the second speckle coding information projected by the light source onto the surface of the target object through the speckle coding pattern to be verified.

[0118] The construction submodule is configured to construct the target speckle image based on the estimated second speckle coding information.

[0119] Optionally, the device further includes:

[0120] The generation module is configured to regenerate the coding pattern at the target position in the speckle coding pattern to be verified if there is a target position in the speckle coding pattern where the coding pattern is invalid.

[0121] Optionally, the device further includes:

[0122] The first determining module is configured to, if there is a target position in the speckle coding pattern that is invalid, determine the degree of invalidity of the coding pattern corresponding to each position based on the decoding result corresponding to each pixel position in the target speckle image;

[0123] The second determining module is configured to determine the weight of a pixel at each location in the target speckle image based on the degree of invalidity, wherein the weight of the pixel is used to characterize the importance of the pixel when calculating depth information;

[0124] The third determining module is configured to determine the depth information of the target speckle image based on the weight of each pixel in the target speckle image.

[0125] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0126] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the method for verifying speckle coding patterns provided in this disclosure.

[0127] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0128] Reference Figure 4 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0129] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the method for verifying speckle-coded patterns. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0130] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0131] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.

[0132] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0133] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0134] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0135] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0136] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0137] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing a method for verifying speckle coding patterns.

[0138] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to complete a method for verifying a speckle coding pattern. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0139] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described method for verifying speckle coding patterns when executed by the programmable device.

[0140] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0141] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for verifying speckle coding patterns, characterized in that, include: Obtain the target speckle image corresponding to the speckle coding pattern to be verified; The target speckle image is decoded to obtain the decoding result corresponding to the position of each pixel in the target speckle image; The validity of the coding pattern corresponding to each pixel position in the speckle coding pattern is verified based on the decoding result corresponding to each pixel position. The decoding result corresponding to the pixel position includes the similarity of the speckle pattern between the target decoding area and other decoding areas within a preset search range at the pixel position; The step of verifying the validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position includes: For each pixel position, in the decoding result corresponding to the pixel position, determine the K decoding regions that are most similar to the speckle image of the target decoding region, and verify the validity of the coding pattern corresponding to the pixel position in the speckle coding pattern according to the position of the K decoding regions, where K is an integer greater than or equal to 2. The step of verifying the validity of the encoding pattern corresponding to the pixel position in the speckle coding pattern based on the positions of the K decoding regions includes: Based on the positions of the K decoding regions, determine whether there exist any two decoding regions whose distance is less than or equal to a preset threshold; If there are any two decoding regions with a distance less than or equal to a preset threshold, then the encoding pattern corresponding to the pixel position in the speckle coding pattern is determined to be invalid. The step of determining whether there exist any two decoding regions whose distance is less than or equal to a preset threshold based on the positions of the K decoding regions includes: In each of the K decoding regions, the coordinates of the center pixel are determined. Based on the coordinates of the center pixel of the K decoding regions, determine whether there are any two other decoding regions whose distance is less than or equal to a preset threshold.

2. The method according to claim 1, characterized in that, The step of acquiring the target speckle image corresponding to the speckle coding pattern to be verified includes: Obtain the first speckle coding information when the light source passes through the speckle coding pattern to be verified; Based on the first speckle coding information and the preset loss function, the second speckle coding information of the light source projected onto the surface of the target object through the speckle coding pattern to be verified is estimated. The target speckle image is constructed based on the estimated second speckle coding information.

3. The method according to claim 1 or 2, characterized in that, The method further includes: If there is an invalid target position in the speckle coding pattern, then the coding pattern at the target position is regenerated in the speckle coding pattern to be verified.

4. The method according to claim 1 or 2, characterized in that, The method further includes: If there is an invalid target position in the speckle coding pattern, the invalidity degree of the coding pattern corresponding to each position is determined according to the decoding result corresponding to each pixel position in the target speckle image; Based on the degree of invalidity, the weight of each pixel at each location in the target speckle image is determined, wherein the weight of the pixel is used to characterize the importance of the pixel when calculating depth information; The depth information of the target speckle image is determined based on the weight of each pixel in the target speckle image.

5. A device for verifying speckle coding patterns, characterized in that, include: The acquisition module is configured to acquire the target speckle image corresponding to the speckle coding pattern to be verified; The decoding module is configured to decode the target speckle image to obtain the decoding result corresponding to the position of each pixel in the target speckle image; The verification module is configured to verify the validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position. The decoding result corresponding to the pixel position includes the similarity of the speckle pattern between the target decoding area and other decoding areas within a preset search range at the pixel position; The verification module includes: The first determining submodule is configured to, for each pixel position, determine the K most similar decoding regions to the speckle image of the target decoding region from the decoding results corresponding to the pixel position, and verify the validity of the encoding pattern corresponding to the pixel position in the speckle encoding pattern according to the positions of the K decoding regions, where K is an integer greater than or equal to 2. The first determining submodule includes: The second determining submodule is configured to determine, based on the positions of the K decoding regions, whether there exist any two decoding regions whose distance is less than or equal to a preset threshold. The third determining submodule is configured to determine that if there are any two decoding regions with a distance less than or equal to a preset threshold, the encoding pattern corresponding to the pixel position in the speckle encoding pattern is invalid. The second determining submodule includes: The fourth determining submodule is configured to determine the coordinates of the center pixel in each of the K decoding regions. The fifth determining submodule is configured to determine, based on the coordinates of the center pixel of the K decoding regions, whether there are any two other decoding regions whose distance is less than or equal to a preset threshold.

6. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Obtain the target speckle image corresponding to the speckle coding pattern to be verified; The target speckle image is decoded to obtain the decoding result corresponding to the position of each pixel in the target speckle image; The validity of the coding pattern corresponding to each pixel position in the speckle coding pattern is verified based on the decoding result corresponding to each pixel position. The decoding result corresponding to the pixel position includes the similarity of the speckle pattern between the target decoding area and other decoding areas within a preset search range at the pixel position; The step of verifying the validity of the encoding pattern corresponding to each pixel position in the speckle coding pattern based on the decoding result corresponding to each pixel position includes: For each pixel position, in the decoding result corresponding to the pixel position, determine the K decoding regions that are most similar to the speckle image of the target decoding region, and verify the validity of the coding pattern corresponding to the pixel position in the speckle coding pattern according to the position of the K decoding regions, where K is an integer greater than or equal to 2. The step of verifying the validity of the encoding pattern corresponding to the pixel position in the speckle coding pattern based on the positions of the K decoding regions includes: Based on the positions of the K decoding regions, determine whether there exist any two decoding regions whose distance is less than or equal to a preset threshold; If there are any two decoding regions with a distance less than or equal to a preset threshold, then the encoding pattern corresponding to the pixel position in the speckle coding pattern is determined to be invalid. The step of determining whether there exist any two decoding regions whose distance is less than or equal to a preset threshold based on the positions of the K decoding regions includes: In each of the K decoding regions, the coordinates of the center pixel are determined. Based on the coordinates of the center pixel of the K decoding regions, determine whether there are any two other decoding regions whose distance is less than or equal to a preset threshold.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 4.

Images