A window determination method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a window determination method, comprising: traversing a preset speckle image through a kth alternative window in L alternative windows; wherein the sizes of different alternative windows are different, k and L are positive integers, and 1<=k<=L; determining an information parameter of speckles in the kth alternative window when the kth alternative window is located in different regions of the preset speckle image, wherein the information parameter comprises: average information amount and / or information difference amount; determining a target window from the L alternative windows according to the information parameter; wherein the target window is used to traverse a target speckle image to obtain target depth information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of information processing, and in particular, to a window determination method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of imaging technology, the quality of images obtained when images are collected by an image collection module is higher and higher. More and more terminal devices have image collection functions, and images can be generated by an image collection module in a terminal device to realize image collection. In order to obtain higher-quality images, the image collection module can generate images with depth information. For example, an image collection module that images according to the principle of structured light imaging can obtain depth information of a current shooting scene, and the image collection module can generate images with depth information according to the depth information. SUMMARY

[0003] The present disclosure provides a window determination method and device, an electronic device, and a storage medium.

[0004] In a first aspect of the embodiments of the present disclosure, a window determination method is provided, including: traversing a preset speckle image through a kth candidate window in L candidate windows; wherein the sizes of different candidate windows are different, k and L are positive integers, and 1≤k≤L; determining an information parameter of speckles in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image, wherein the information parameter includes: an average information amount and / or an information difference amount; determining a target window from the L candidate windows according to the information parameter; wherein the target window is used to traverse a target speckle image to obtain target depth information.

[0005] In one embodiment, when the preset speckle image is traversed, the kth candidate window traverses N different regions in the preset speckle image; the method includes: determining a probability density of a gray scale m in the kth candidate window in an ith region when the kth candidate window is located in the ith region according to a gray scale of pixels included in the kth candidate window; wherein 1≤i≤N.

[0006] In one embodiment, the determination of the information parameter of speckles in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image includes: determining the average information amount according to an entropy of the probability density of each gray scale corresponding to the kth candidate window in N regions.

[0007] In an embodiment, the determining the information parameter of the speckle in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image comprises: determining the information difference amount according to the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the ith region and the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the jth region; wherein the jth region is a region other than the ith region in the N regions.

[0008] In an embodiment, when the information parameter comprises the average information amount and the information difference amount, the determining the target window from the L candidate windows according to the information parameter comprises: determining a first reference value according to the average information amount and a first weight of the average information amount; determining a second reference value according to the information difference amount and a second weight of the information difference amount; and determining the target window according to the first reference value and the second reference value.

[0009] In an embodiment, the determining the target window according to the first reference value and the second reference value comprises: determining the candidate window with the maximum sum of the first reference value and the second reference value as the target window.

[0010] In an embodiment, the determining the target window from the L candidate windows according to the information parameter comprises: determining the target window from the L candidate windows according to the number of different regions traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information amount, and the information difference amount.

[0011] In a second aspect, the embodiment of the present disclosure provides a window determination apparatus, comprising: a traversal module configured to traverse a preset speckle image through a kth candidate window in L candidate windows; wherein the different candidate windows are different in size, k and L are positive integers, and k is less than or equal to L; an average information amount determination module configured to determine an information parameter of a speckle in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image, wherein the information parameter comprises an average information amount and / or an information difference amount; and a target window determination module configured to determine a target window from the L candidate windows according to the information parameter; wherein the target window is used to traverse a target speckle image to obtain target depth information.

[0012] In a third aspect, the embodiment of the present disclosure provides an electronic device, comprising:

[0013] A processor and a memory for storing executable instructions capable of running on the processor, wherein when the processor is used to run the executable instructions, the executable instructions perform the method described in any of the above embodiments.

[0014] In a fourth aspect, the embodiments of the present disclosure provide a non-transitory computer-readable storage medium, which stores computer executable instructions. When the computer executable instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0015] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects.

[0016] The embodiments of the present disclosure traverse the preset speckle image through the kth alternative window in the L alternative windows of different sizes, k is the first window to the Lth window, that is, traverse the preset speckle image through each window in the L windows. Then determine the information parameters of the speckles in the kth alternative window when the kth alternative window is located in different regions of the preset speckle image, the information parameters include: average information amount and / or information difference amount. Then according to the information parameters corresponding to each window, determine the target window from the L alternative windows, which can be used to traverse the target speckle image to obtain the target depth information.

[0017] By determining the target window according to the information parameters of the alternative window when traversing the preset speckle image, the target window can be used to traverse the corresponding target speckle image in the actual shooting scene, so as to obtain the target depth information corresponding to the actual shooting scene, thereby facilitating the generation of a depth image according to the depth information. Through this method, a more accurate window can be obtained, thereby obtaining more accurate depth information, and reducing the problem of low universality and accuracy caused by manually determining the window according to experience.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0020] Figure 1 is a schematic diagram of a structured light imaging structure according to an exemplary embodiment;

[0021] Figure 2 is a schematic diagram of a specific application of a window according to an exemplary embodiment;

[0022] Figure 3is a schematic diagram of obtaining depth information according to an example embodiment;

[0023] Figure 4 is a flowchart of a method for determining a window according to an example embodiment;

[0024] Figure 5 is a schematic diagram of determining a target window according to an example embodiment;

[0025] Figure 6 is a structural schematic diagram of a window determining apparatus according to an example embodiment;

[0026] Figure 7 is a block diagram of a terminal device according to an example embodiment. DETAILED DESCRIPTION

[0027] The example embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, unless otherwise indicated, like numbers in the different drawings represent the same or similar elements. The following example embodiments described in the detailed description are not meant to be an all-inclusive description of all aspects of the disclosure. Rather, they are merely example embodiments consistent with some aspects of the disclosure as detailed in the appended claims.

[0028] In an imaging system, imaging can be performed by a structured light imaging principle to obtain depth information. Referring to Figure 1 is a schematic diagram of a structured light imaging structure, including a speckle emitting end and a speckle receiving end. The speckle emitting end can be a speckle emitter, and the speckle receiving end can be an image sensor for image acquisition. The speckle emitting end sends speckles to a preset object, and the speckle receiving end receives speckles reflected back by a target object to obtain a speckle reference image. The preset object can be a whiteboard, which can be different from the objects in an actual shooting scene and used to obtain a speckle reference image.

[0029] For example, the speckle emitter projects specially coded speckle patterns onto the surface of the whiteboard. When the speckle patterns are reflected by the surface of the object, different deformations occur with different distances of the object. The image sensor can acquire the deformed patterns to obtain a speckle reference image, which is then saved. The patterns acquired by the image sensor can be used to determine depth information.

[0030] Then, the above method is used to shoot a target object in an actual shooting scene. The speckle emitter emits specially coded speckle patterns to the target object, and then the image sensor can acquire the deformed patterns to obtain a speckle target image.

[0031] According to the speckle reference image and the speckle target image, a depth image can be obtained, and the deformation of each pixel in the speckle reference image and the speckle target image is calculated by the structured light algorithm to obtain corresponding parallax, so as to further obtain depth information. For example, the speckle reference image and the speckle target image are matched by a window matching method to obtain depth image information.

[0032] Reference Figure 2 is a schematic diagram of a specific application of a window, Figure 2 part (a) of is a schematic diagram of an application, Figure 2 part (b) of is Figure 2 another form of (a). Reference Figure 3 is a schematic diagram of matching the speckle reference image and the speckle target image by a window matching method to obtain depth information. Figure 3 part (a) of is a speckle reference image, Figure 3 part (b) of is a speckle target image. The speckle can be encoded speckle after encoding, and the window can decode the speckle target image obtained by the speckle receiver to obtain depth information.

[0033] In combination with Figure 2 and Figure 3 , after the speckle emitter sends random speckle, the speckle receiver obtains the speckle target image, and when determining the deformation of the center pixel of the speckle target image, the center pixel is taken as the center to match the corresponding position in the speckle target image through the window. For example, the window is a 25x25 pixel image block, and then based on the window matching, the pixel regions at the same positions in the speckle reference image and the speckle target image are matched through the window, that is, the region of the window in the speckle reference image and the region of the window in the speckle target image, and by calculating the offset of the pixel information of the window in the two images, the parallax of the region where the window is located can be obtained, so that the corresponding depth information can be determined according to the principle of triangulation combined with the physical parameters of the image acquisition module.

[0034] In this process, the size of the window, that is, the size of the image block, will affect the parallax, and thus can affect the determined depth information. Generally, the size of the window can be determined according to empirical values or experimental values, resulting in inaccurate window size and low universality, thereby affecting the determination of the depth information and reducing the accuracy of the depth information.

[0035] Reference Figure 4 is a flowchart of a method for determining a window according to the technical scheme, and the method comprises the following steps:

[0036] In step S100, the kth candidate window in L candidate windows is used to traverse a preset speckle image; wherein the sizes of different candidate windows are different, k and L are positive integers, and 1≤k≤L.

[0037] In step S200, information parameters of speckles in the kth candidate window are determined when the kth candidate window is located in different regions in the preset speckle image, wherein the information parameters include average information amount and / or information difference amount.

[0038] In step S300, a target window is determined from the L candidate windows according to the information parameters, wherein the target window is used to traverse a target speckle image to obtain target depth information.

[0039] For step S100, L candidate windows are determined, and then each of the L windows is used to traverse the preset speckle image respectively, so that information parameters corresponding to each window can be obtained, and a target window can be determined from the information parameters. The L windows can be L windows determined from a plurality of windows with different sizes, for example, L candidate windows determined from historical windows used before, and the L windows have different sizes. The size of a window can be represented by pixels, for example, the size of window 1 is 2*2 pixels, the size of window 2 is 3*3 pixels, the size of window 3 is 5*5 pixels, and so on.

[0040] In an embodiment, the L windows can also be arranged in a sequence to form a window sequence, and each window in the window sequence has a different size. For example, Set W = {W1, W2, …, WL} L}, Set W represents a window sequence, and the window sequence includes L windows, W L represents the Lth window, and the size of the ith window is different from that of the jth window, which can be represented by W i ≠ W j , i ≠ j. The value of L can be determined according to actual application scenarios, and L can be different in different application scenarios.

[0041] The preset speckle image is traversed by the kth candidate window, 1 ≤ k ≤ L, and k is the first candidate window to the kth candidate window. The kth candidate window can be represented by W k , when k is equal to 1, it means that the preset speckle image is traversed by the first window, when k is equal to 2, it means that the preset speckle image is traversed by the second window, and when k is equal to L, it means that the preset speckle image is traversed by the Lth window. The preset speckle image is traversed by the first candidate window to the Lth candidate window in the L windows respectively, and each window in the L candidate windows is represented by the kth candidate window.

[0042] The preset speckle image can be a speckle image collected by the speckle receiver after the speckle emitter emits speckles to the preset object and the speckles are reflected by the preset object. The distance between the speckle emitter and the preset object can be x, and the unit of x can be meter, decimeter, centimeter, etc. The value of x can be determined according to the actual shooting scene, for example, 2 meters, 5 meters, 10 meters, 50 meters, 20 centimeters or 50 centimeters, etc. The preset object can be a whiteboard, and can also be other preset objects determined according to actual needs, which are not limited here.

[0043] In an embodiment, the preset speckle images of the L alternative window traversals are the same preset speckle image, that is, they are all speckle images collected by the speckle receiver after the speckle emitter emits speckles to the same preset object at a distance of x.

[0044] In another embodiment, after the window in the L alternative windows is updated, and / or after the actual use demand changes, such as after the shooting scene changes, the size of the distance x between the speckle emitter and the preset object can be adjusted.

[0045] In an embodiment, when the alternative window traverses the preset speckle image, it can be traversed in a non-overlapping manner, that is, the area covered by the alternative window each time does not overlap.

[0046] When the alternative window traverses the preset speckle image, the alternative window traverses the preset speckle image according to the size of the alternative window. When the size of the alternative window is different, the number of areas traversed by the alternative window when traversing the preset speckle image can be different. When the preset speckle image is unchanged, the number of different areas in the preset speckle image traversed by the larger alternative window when traversing the preset speckle image can be smaller, and the number of movements can be smaller. The number of different areas in the preset speckle image traversed by the smaller alternative window when traversing the preset speckle image can be larger, and the number of movements can be larger.

[0047] For example, the size of the preset speckle image is 1024*1440 pixels. When the first alternative window with a size of 3*3 traverses the preset speckle image, the number of movements of the first alternative window is U1, that is, there are U1 areas in the preset speckle image with the size of the first alternative window. When the second alternative window with a size of 5*5 traverses the preset speckle image, the number of movements of the second alternative window is U2, that is, there are U2 areas in the preset speckle image with the size of the second alternative window. At this time, U1 is greater than U2.

[0048] For step S200, when the alternative window traverses the preset speckle image, the alternative window needs to move in the preset speckle image to cover different areas of the preset speckle image. Taking the kth alternative window as an example, the information parameters corresponding to each alternative window in the L alternative windows can be determined through this step.

[0049] The information parameter of the speckle in the kth candidate window is determined when the kth candidate window is located in different regions of the preset speckle image. The information parameter includes: average information amount and / or information difference amount. When the kth candidate window is located in each region of the preset speckle image, the kth candidate window covers a part of pixels in the preset speckle image. The information amount of the speckle in the kth candidate window can be determined according to the pixel information of the preset speckle image, for example, the gray scale information of the pixel. The information amount of the speckle in the kth candidate window can represent the distribution of the speckle in the kth candidate window, which can be determined according to the gray scale information of the pixel.

[0050] After determining the information amount of the speckle in the kth candidate window when the kth candidate window is located in each region of the preset speckle image, the average information amount of the speckle in the kth candidate window can be determined after the kth candidate window traverses the preset speckle image. Similarly, the information difference amount between the information amount of the speckle in the kth candidate window when the kth candidate window is located in each region of the preset speckle image can also be determined.

[0051] For example, when k is equal to 1, the first candidate window with a size of 3*3 pixels traverses the preset speckle image. The preset speckle image is an image with a size of 9*9 pixels. The preset speckle image has nine non-overlapping regions with the size of the first candidate window, which are the first region to the ninth region. The first candidate window needs to traverse the nine regions in the preset speckle image when traversing the preset speckle image. The traversal order can not be limited from the first region to the ninth region. The first information parameter of the speckle in the first candidate window is determined after the first candidate window traverses the first region to the ninth region.

[0052] Specifically, it can include: determining the first information amount in the first candidate window when the first candidate window is in the first region, determining the second information amount in the first candidate window when the first candidate window is in the second region, and determining the ninth information amount in the first candidate window when the first candidate window is in the ninth region. According to the nine information amounts, the average information amount of the speckle in the first candidate window when the first candidate window traverses the preset speckle image can be determined. At the same time, the information difference amount between the corresponding information amounts when the first candidate window is in different regions can also be determined, for example, the difference amount between the first information amount and the second information amount to the ninth information amount.

[0053] Similarly, for the second candidate window to the Lth candidate window in the L candidate windows, the corresponding information amount of each candidate window can be determined.

[0054] For step S300, after determining the information parameters corresponding to each candidate window respectively, the target window is determined from the L candidate windows according to the information parameters corresponding to each candidate window. In actual application scenarios, the target window determined can be used to traverse the target speckle image received by the speckle receiver during shooting, so as to obtain the corresponding target depth information.

[0055] For example, the candidate window with the maximum sum of information parameters is determined, and then the candidate window is taken as the target window. Since the greater the average amount of information in the candidate window, the more speckle information in the window, more speckle information can be obtained through the candidate window. The greater the information difference, the greater the difference in the amount of information corresponding to the different regions of the preset speckle image when the candidate window traverses the preset speckle image, so that the depth information can be better obtained. The target window determined in combination with the average amount of information and the information difference is more accurate in the depth information obtained when traversing the target speckle image.

[0056] By determining the target window according to the information parameters of the candidate window when traversing the preset speckle image, the target window can be used to traverse the corresponding target speckle image in the actual shooting scene, so as to obtain the target depth information corresponding to the actual shooting scene, thereby facilitating the generation of a depth image according to the depth information. Through this method, a more accurate window can be obtained, so that more accurate depth information can be obtained, and the problem of low universality and accuracy caused by manual determination of the window according to experience is reduced.

[0057] In another embodiment, when traversing the preset speckle image, the kth candidate window traverses N different regions in the preset speckle image.

[0058] The method further comprises: determining the probability density of the gray scale m in the kth candidate window in the i th region according to the gray scale of the pixels contained in the kth candidate window in the i th region; wherein 1≤i≤N.

[0059] The kth candidate window needs to traverse N different regions when traversing the preset speckle image, i.e. the preset speckle image has N regions of the size of the kth candidate window, and the kth candidate window can complete the traversal of the preset speckle image after traversing the N different regions. For example, the preset speckle image has nine non-overlapping regions of the size of the first candidate window, i.e. first region to ninth region. The first candidate window needs to traverse nine regions in the preset speckle image when traversing the preset speckle image.

[0060] When the kth candidate window traverses the ith region, the gray scale information of each pixel in the kth candidate window can be determined, and then the probability density of each gray scale can be determined according to the gray scale information of each pixel in the kth candidate window. For example, the probability density of each gray scale in the kth candidate window in the third region is determined according to the gray scale of the pixel contained in the first candidate window when the first candidate window is located in the third region. In this embodiment, m is used to represent each gray scale, and the value of m can be determined according to actual needs, for example, 0≤m≤255.

[0061] For example, the probability density of the pixel with the gray scale of 0 is the first probability density, the probability density of the pixel with the gray scale of 1 is the second probability density, and so on, and the probability density of the pixel with the gray scale of 255 is the two hundred and fifty-sixth probability density.

[0062] In this way, the probability density of the gray scale of each pixel in the kth candidate window when the kth candidate window is located in each of the first to ninth regions can be determined.

[0063] The process of specifically determining the gray scale information and determining the probability density according to the gray scale information is not described in detail here. The gray scale information can be determined by recognition or detection, and the probability density of the gray scale can be determined by a probability density algorithm. The probability density of each gray scale can also be determined in the form of a probability density histogram.

[0064] In another embodiment, in step S200, the information parameters of the speckle in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image are determined, including:

[0065] The average information amount is determined according to the entropy of the probability density of each corresponding gray scale of the kth candidate window in the N regions.

[0066] This embodiment is an embodiment for determining the average information amount. After the probability density of each corresponding gray scale of the kth candidate window in each region is determined, the entropy of the probability density of each corresponding gray scale of the kth candidate window in each region can be determined, and then the average information amount of the speckle in the window of the kth candidate window in each region can be determined according to the entropy of the probability density of each corresponding gray scale of the kth candidate window in each region.

[0067] For example, represents that the kth candidate window is located in the ith region in the preset speckle image, represents the probability density of the gray scale of m of the pixel in the kth candidate window when the kth candidate window is located in the ith region in the preset speckle image, and the entropy of the probability density of the gray scale of each pixel in the kth candidate window when the kth candidate window is located in the ith region in the preset speckle image can be determined by formula (1).

[0068]

[0069] i.e. the entropy of the probability density of the gray scale of each pixel in the kth candidate window when the kth candidate window is located in the ith region in the preset speckle image.

[0070] The entropy of the probability density of the gray scale of each pixel in the kth candidate window when the kth candidate window is located in each region in the preset speckle image can be determined by the above formula (1), and the entropy of the probability density corresponding to each of the N regions. For example, when N is equal to 9, the entropy of the probability density corresponding to each of the 9 regions with i equal to 1 to i equal to 9 can be determined.

[0071] Then, the average information amount is determined according to the entropy of the probability density of each gray scale corresponding to the kth candidate window in the N regions. The average information amount can be the arithmetic mean of the entropy of the probability density of each gray scale corresponding to the kth candidate window in the N regions.

[0072] For example, the average information amount can be determined by formula (2).

[0073]

[0074] E k The average information amount of the speckle in the kth candidate window when the kth candidate window is in the N regions, i.e. the arithmetic mean of the entropy of the probability density of each gray scale corresponding to the kth candidate window in the N regions.

[0075] The average information amount corresponding to each of the L candidate windows in the first candidate window to the Lth candidate window can be determined by the formula (1) and the formula (2).

[0076] In another embodiment, in step S200, the information parameter of the speckle in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image is determined, including:

[0077] According to the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the ith region, and the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the jth region, the information difference amount is determined. The jth region is a region in the N regions other than the ith region.

[0078] The embodiment is for determining the information difference amount. After determining the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the ith region, the difference degree of the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the ith region and the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in other regions can be determined according to the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in different regions. Then, the average difference degree of the difference degrees corresponding to different gray scales is determined according to the difference degree, and then the target difference degrees when i is different values, i.e., the information difference amount, can be determined according to the average difference degree.

[0079] In the N regions, the regions other than the ith region can be represented by the jth region.

[0080] For example, the difference degree of the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the ith region and the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in other regions, and the average difference degree of the difference degrees corresponding to different gray scales can be determined according to formula (3).

[0081]

[0082] P (m | i, k) represents the probability density of the gray scale m of the pixels in the kth candidate window when the kth candidate window is located in the ith region. P (m | j, k) represents the probability density of the gray scale m of the pixels in the kth candidate window when the kth candidate window is located in the jth region. P (m | i, k) represents the probability density of the gray scale m of the pixels in the kth candidate window when the kth candidate window is located in the ith region. P (m | i, k) represents the probability density of the gray scale m of the pixels in the kth candidate window when the kth candidate window is located in the ith region.

[0083] The information difference amount can be determined by formula (4),

[0084]

[0085] P (m | i, k) represents the probability density of the gray scale m of the pixels in the kth candidate window when the kth candidate window is located in the ith region.

[0086] For example, when i equals 1 and N equals 9, the average of the cross-entropy of the respective gray scale of the pixel in the kth alternative window in the first region and the respective gray scale of the pixel in the kth alternative window in the other eight regions can be determined by formula (3). The average of the cross-entropy corresponding to i equaling 1 to 9 can be determined by formula (4), and then the information difference quantity can be determined according to the average of the nine averages. For example, the first average corresponding to the cross-entropy when i equals 1, the second average corresponding to the cross-entropy when i equals 1, and the ninth average corresponding to the cross-entropy when i equals 9, and then the information difference quantity can be determined according to the nine averages.

[0087] For each alternative window, the corresponding information difference quantity can be determined.

[0088] In another embodiment, with reference to Figure 5 For a schematic diagram of determining a target window, when the information parameters include the average information quantity and the information difference quantity, the step S300 comprises: determining the target window from the L alternative windows according to the information parameters, comprising:

[0089] The step S301 comprises: determining a first reference value according to the average information quantity and a first weight of the average information quantity.

[0090] The step S302 comprises: determining a second reference value according to the information difference quantity and a second weight of the information difference quantity.

[0091] The step S303 comprises: determining the target window according to the first reference value and the second reference value.

[0092] The average information quantity and the information difference quantity have a weight respectively, the first reference value is determined according to the average information quantity and the first weight of the average information quantity, the second reference value is determined according to the information difference quantity and the second weight of the information difference quantity, and the target window is determined by combining the first reference value and the second reference value. The first weight of the average information quantity and the second weight of the information difference quantity can be determined according to actual needs. For example, the alternative window with the maximum sum of the first reference value and the second reference value can be determined as the target window.

[0093] The target window can be determined from multiple dimensions by combining the average information quantity and the information difference quantity, which reduces the limitations caused by determining the target window according to a single indicator, thereby improving the accuracy of determining the target window and the accuracy of the depth information.

[0094] In another embodiment, the step S300 comprises: determining the target window from the L alternative windows according to the information parameters, comprising:

[0095] The target window is determined according to the first reference value, and the alternative window with the maximum first reference value is determined as the target window.

[0096] The target window is determined according to the second reference value, and a candidate window with the maximum second reference value is determined as the target window.

[0097] In another embodiment, the step S300 of determining the target window from the L candidate windows according to the information parameter comprises:

[0098] The target window is determined from the L candidate windows according to the number of different regions traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information amount, and the information difference amount.

[0099] On the basis of the average information amount and the information difference amount, the target window is determined in combination with the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image. The target window can be determined in combination with the average information amount and the information difference amount according to the product of the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image.

[0100] The product of the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image can be mapped to the interval of 0 to 1. Since the values of the average information amount and the information difference amount are in the interval of 0 to 1, the target window can be determined conveniently.

[0101] When the size of the preset speckle image is unchanged, the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image affect the operation amount (operation times) of obtaining the target depth information, that is, the size of the window affects the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image, so the size of the window is related to the operation amount, thereby affecting the efficiency of the captured image.

[0102] On the basis of the average information amount and the information difference amount, the target window that is more matched can be determined in combination with the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image.

[0103] When the size of the preset speckle image and the size of the candidate window are known, the number N of regions traversed by the candidate window in the preset speckle image and the number of pixels in the candidate window in each region when the candidate window traverses the preset speckle image can be determined.

[0104] The operation amount of the kth candidate window traversing the preset speckle image can be represented by formula (5):

[0105]

[0106] represents the operation amount of the kth alternative window for traversing the preset speckle image to obtain the average information amount and the information difference amount, C k is the operation amount after normalization. and C k The mapping relationship between and C can be determined according to actual requirements.

[0107] According to this embodiment, the operation amount corresponding to each alternative window can be determined.

[0108] The third reference value is determined according to the operation amount and the third weight of the operation amount, and the target window is determined in combination with the first reference value, the second reference value and the third reference value. For example, in combination with the sum of the first reference value, the second reference value and the third reference value, the alternative window with the maximum sum value is determined as the target window.

[0109] For example, the sum of the first reference value, the second reference value and the third reference value can be determined by formula (6).

[0110] S k = w E *E k +w D *D k +w c *C k (6)

[0111] w E represents the first weight of the average information amount E k corresponding to the kth alternative window, w D represents the second weight of the information difference amount D k corresponding to the kth alternative window, w c represents the third weight of the operation amount C k corresponding to the kth alternative window, S k represents the sum of the first reference value, the second reference value and the third reference value obtained by the kth alternative window.

[0112] In another embodiment, with reference Figure 6 , it is a structural schematic diagram of a window determination device, which comprises:

[0113] The traversal module 1 is configured to traverse a preset speckle image through a kth alternative window in L alternative windows; wherein the sizes of different alternative windows are different, k and L are positive integers, and k is less than or equal to L.

[0114] The information parameter determination module 2 is configured to determine an information parameter of speckles in the kth candidate window when the kth candidate window is located in different regions in the preset speckle image, wherein the information parameter comprises an average information amount and / or an information difference amount.

[0115] The target window determination module 3 is configured to determine a target window from the L candidate windows according to the information parameter, wherein the target window is used to traverse a target speckle image to obtain target depth information.

[0116] In another embodiment, when the preset speckle image is traversed, the kth candidate window traverses N different regions in the preset speckle image.

[0117] The device further comprises:

[0118] The probability density determination module is configured to determine a probability density of a gray scale m in the kth candidate window in the ith region according to a gray scale of pixels contained in the kth candidate window when the kth candidate window is located in the ith region, wherein 1≤i≤N.

[0119] In another embodiment, the information parameter determination module 2 is further configured to determine the average information amount according to an entropy of the probability density of each gray scale corresponding to the N regions in the kth candidate window.

[0120] In another embodiment, the information parameter determination module 2 is further configured to determine the information difference amount according to the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the ith region and the probability density of the gray scale m in the kth candidate window when the kth candidate window is located in the jth region.

[0121] The jth region is a region other than the ith region in the N regions.

[0122] In another embodiment, the target window determination module 3 comprises:

[0123] The first reference value determination unit is configured to determine a first reference value according to the average information amount and a first weight of the average information amount.

[0124] The second reference value determination unit is configured to determine a second reference value according to the information difference amount and a second weight of the information difference amount.

[0125] The target window determination unit is configured to determine the target window according to the first reference value and the second reference value.

[0126] In another embodiment, the target window determining unit is specifically configured to: determine the target window as the candidate window with the maximum sum of the first reference value and the second reference value.

[0127] In another embodiment, the target window determining module 3 is further configured to:

[0128] The target window is determined from the L candidate windows according to the number of different areas traversed by the kth candidate window in the target speckle image, the number of pixels contained in the kth candidate window, the average information amount and the information difference amount.

[0129] In another embodiment, an electronic device is also provided, comprising:

[0130] A processor and a memory for storing executable instructions capable of running on the processor, wherein:

[0131] When the processor is configured to run the executable instructions, the executable instructions perform the method of any of the above embodiments.

[0132] In another embodiment, a non-transitory computer readable storage medium is also provided, the computer readable storage medium stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the method of any of the above embodiments.

[0133] It should be noted that the "first" and "second" in the embodiments of the present disclosure are only for convenience of expression and distinction, and have no other specific meaning.

[0134] Figure 7 is a block diagram of a terminal device according to an example embodiment. The terminal device can be, for example, a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0135] Referring to Figure 7 , the terminal device can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0136] The processing component 802 generally controls the overall operations of the terminal device, such as operations associated with displaying, making a phone call, conducting data communications, operating a camera, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete the steps of the methods described above, in whole or in part. Moreover, the processing component 802 can include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0137] The memory 804 is configured to store various types of data to support the operations of the terminal device. Examples of these data include instructions for any application or methods operating on the terminal device, contact data, phonebook data, messages, pictures, videos, and so on. The memory 804 can be realized by any type of volatile or non-volatile storage devices, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disc, or optical disc.

[0138] The power component 806 provides power to the various components of the terminal device. The power component 806 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the terminal device.

[0139] The multimedia component 808 includes a screen providing an output interface between the terminal device and the user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. The front and rear cameras can receive external multimedia data when the terminal device is in an operating mode, such as a shooting mode or a video mode. Each of the front and rear cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0140] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive an external audio signal when the terminal device is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0141] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0142] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the terminal device. For example, the sensor component 814 can detect an open / closed position of the terminal device, relative positioning of components, such as a display and a keypad of the terminal device, a change in position of the terminal device or a component of the terminal device, the presence or absence of user contact with the terminal device, the orientation or acceleration / deceleration of the terminal device, and a temperature change of the terminal device. The sensor component 814 can include a proximity sensor configured to detect the presence of a nearby object without any physical touch. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0143] The communication component 816 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0144] In an example embodiment, the terminal device can be implemented with one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components, for performing the above-described methods.

[0145] Other embodiments of the disclosure will be apparent to those of ordinary skill in the art from a consideration of the specification and practice of the disclosure disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0146] It is to be understood that the disclosure is not limited to the precise construction described above and shown in the attached drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the disclosure is limited only by the claims appended hereto.

Claims

1. A method for determining a window, characterized in that, include: The preset speckle image is traversed through the k-th candidate window among L candidate windows; wherein the sizes of the different candidate windows are different, and k and L are positive integers, 1≤k≤L; When the k-th candidate window is located in different regions of the preset speckle image, the information parameters of the speckle within the k-th candidate window are determined, wherein the information parameters include: average information content and information difference; the information difference is determined based on the difference in speckle information content of the k-th candidate window in each region when the k-th candidate window is located in different regions of the preset speckle image, and the information content is determined based on the pixel information of the preset speckle image covered by the k-th candidate window; A first reference value is determined based on the average information content and a first weight of the average information content; A second reference value is determined based on the information difference amount and the second weight of the information difference amount; Based on the first reference value and the second reference value, a target window is determined from the L candidate windows; wherein, the target window is used to traverse the target speckle image to obtain target depth information; the preset object corresponding to the preset speckle image is different from the target object corresponding to the target speckle image.

2. The method according to claim 1, characterized in that, When traversing the preset speckle image, the kth candidate window traverses N different regions in the preset speckle image; The method includes: Based on the grayscale of the pixels contained in the k-th candidate window when it is located in the i-th region, determine the probability density of grayscale m in the k-th candidate window in the i-th region; where 1≤i≤N.

3. The method according to claim 2, characterized in that, When the k-th candidate window is located in a different region of the preset speckle image, the speckle information parameters within the k-th candidate window include: The average information content is determined based on the entropy of the probability density of each gray level corresponding to the k-th candidate window in the N regions.

4. The method according to claim 2, characterized in that, When the k-th candidate window is located in a different region of the preset speckle image, the speckle information parameters within the k-th candidate window include: The information difference is determined based on the probability density of gray level m in the k-th candidate window when the k-th candidate window is located in the i-th region, and the probability density of gray level m in the k-th candidate window when the k-th candidate window is located in the j-th region. Wherein, the j-th region is the region other than the i-th region among the N regions.

5. The method according to claim 1, characterized in that, The step of determining the target window from the L candidate windows based on the first reference value and the second reference value includes: The candidate window that maximizes the sum of the first reference value and the second reference value is selected as the target window.

6. The method according to claim 1, characterized in that, The step of determining the target window from the L candidate windows based on the first reference value and the second reference value includes: The target window is determined from the L candidate windows based on the number of different regions traversed by the k-th candidate window in the target speckle image, the number of pixels contained in the k-th candidate window, the first reference value, and the second reference value.

7. A window determining device, characterized in that, include: The traversal module is used to traverse the preset speckle image through the k-th candidate window among L candidate windows; wherein the different candidate windows have different sizes, k and L are positive integers, and k is less than or equal to L; An information parameter determination module is used to determine the information parameters of the speckle within the k-th candidate window when the k-th candidate window is located in different regions of the preset speckle image. The information parameters include: average information content and information difference. The information difference is determined based on the difference in speckle information content in each region of the k-th candidate window when it is located in different regions of the preset speckle image. The information content is determined based on the pixel information of the preset speckle image covered by the k-th candidate window. The target window determination module is used to determine a first reference value based on the average information content and a first weight of the average information content; determine a second reference value based on the information difference and a second weight of the information difference; and determine a target window from L candidate windows based on the first reference value and the second reference value; wherein the target window is used to traverse the target speckle image to obtain target depth information; and the preset object corresponding to the preset speckle image is different from the target object corresponding to the target speckle image.

8. An electronic device, characterized in that, include: A processor and a memory for storing executable instructions capable of running on the processor, wherein: When the processor is used to run the executable instructions, the executable instructions perform the method described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by a processor, implement the method described in any one of claims 1 to 6.

Images