Image Processing Method, Apparatus, Electronic Device, and Readable Storage Medium

Abstract

The present disclosure provides an image processing method, apparatus, electronic device, and readable storage medium. The method includes: obtaining first window sizes of a plurality of first windows of a depth camera according to an image captured by the depth camera; adjusting the first window sizes of the plurality of first windows to a reference window size to obtain a plurality of second windows; determining a target window size based on the occurrence frequencies of different pixel gray values among the plurality of second windows according to the second window size, and performing feature matching processing on the image based on the target window size. By adjusting the first window sizes of the depth camera to the reference window size to obtain a plurality of second windows, and determining the target window size according to the second window size based on the occurrence frequencies of different pixel gray values among the plurality of second windows, the present disclosure avoids setting the window size solely relying on empirical values, and improves the setting efficiency of the window size and the accuracy of window matching.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision, and in particular, to an image processing method, apparatus, electronic device, and readable storage medium. Background Art

[0002] With the development of the stereo vision field in computer vision, the application of depth cameras has become increasingly common. Through a device equipped with a depth camera, position perception information within the search radius can be obtained while imaging, which is used to restore the real scene and implement functions such as scene modeling. And the window, as important information for determining image matching and image encoding in stereo vision, the selection of its size can determine the effective degree of the depth camera for obtaining perception information.

[0003] Currently, the selection of the window size by depth cameras is empirical, that is, by using a pre-set window size to match and decode specific scene images, and it is necessary to continuously adjust and optimize the window size to adapt to different scenes. The setting efficiency of the window size is low, and the reliability of the final target window for matching is insufficient. Summary of the Invention

[0004] In view of this, the present disclosure provides an image processing method, apparatus, electronic device, and readable storage medium to at least solve the problem in the related art that the window size cannot be automatically adjusted to the optimal matching state.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, the method including:

[0006] Obtain the first window sizes of a plurality of first windows of the depth camera according to an image captured by the depth camera, where the window is a rectangular area for determining an image feature matching range;

[0007] Adjust the first window sizes of the plurality of first windows to a reference window size to obtain a plurality of second windows, where the reference window size is determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera;

[0008] Based on the occurrence frequencies of different pixel gray values among the plurality of second windows, determine a target window size according to the second window size;

[0009] Perform feature matching processing on the image based on the target window size.

[0010] Combined with any implementation manner of the present disclosure, the reference window size determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera includes:

[0011] Within the search radius of the depth camera, obtain the disparities of the multiple first windows respectively;

[0012] Obtain multiple first windows with disparities less than a first set threshold;

[0013] Determine the smallest window size among the multiple first windows as the reference window size.

[0014] Combined with any embodiment of the present disclosure, the determining the target window size according to the second window size includes:

[0015] In response to the occurrence frequencies of different pixel gray values among the multiple second windows satisfying a set condition, determine the target window size according to the window size of the second window;

[0016] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition, enlarge the multiple second windows until the set condition is satisfied.

[0017] Combined with any embodiment of the present disclosure, the occurrence frequencies of different pixel gray values among the multiple second windows satisfying the set condition includes:

[0018] The degree of dispersion of the gray value information amounts of the multiple second windows is not greater than a second set threshold, and / or;

[0019] Among the gray value cross-entropies between the multiple second windows, the minimum cross-entropy is not less than a third set threshold.

[0020] Combined with any embodiment of the present disclosure, the enlarging the multiple second windows includes:

[0021] Enlarge the window sizes of the multiple second windows according to a set ratio, or;

[0022] Increase the window sizes of the multiple second windows according to a set size.

[0023] Combined with any embodiment of the present disclosure, the in response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition, enlarging the multiple second windows until the set condition is satisfied includes:

[0024] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition and the number of iterations reaching the iteration number threshold, determine the target window size according to the second window size obtained in the last iteration.

[0025] Combined with any embodiment of the present disclosure, the determining the target window size according to the second window size includes:

[0026] When the depth camera is a binocular depth camera, determine the second window size as the target window size;

[0027] When the depth camera is a structured light depth camera, determine the target window size according to the window size of the second window and the number of pixels of the minimum target and the light spot determined by the structured light depth camera in the image.

[0028] According to a second aspect of the embodiments of the present disclosure, there is provided a window adjustment device, the device includes:

[0029] Combined with any implementation manner of the present disclosure, the device includes:

[0030] The first window acquisition module: according to the image captured by the depth camera, acquire the first window sizes of a plurality of first windows of the depth camera, wherein the window is a rectangular area for determining the image feature matching range;

[0031] The second window acquisition module: used to adjust the first window sizes of the plurality of first windows to a reference window size to obtain a plurality of second windows, wherein the reference window size is determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera;

[0032] The target window determination module: used to determine the target window size based on the occurrence frequencies of different pixel gray values among the plurality of second windows according to the second window size;

[0033] Based on the target window size, perform feature matching processing on the image.

[0034] Combined with any implementation manner of the present disclosure, in the second window acquisition module, the reference window size determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera is used for:

[0035] Within the search radius of the depth camera, respectively acquire the disparities of the plurality of first windows;

[0036] Acquire a plurality of first windows whose disparities are less than a first set threshold;

[0037] Determine the smallest window size among the plurality of first windows as the reference window size.

[0038] Combined with any implementation manner of the present disclosure, the second window acquisition module is used to determine the target window size according to the second window size, specifically used for:

[0039] In response to the occurrence frequencies of different pixel gray values among the multiple second windows satisfying a set condition, determine a target window size according to the window size of the second window;

[0040] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition, enlarge the multiple second windows until the set condition is satisfied.

[0041] Combined with any embodiment of the present disclosure, the occurrence frequencies of different pixel gray values among the multiple second windows satisfying the set condition are used for:

[0042] The degree of dispersion of the gray value information amount of the multiple second windows is not greater than a second set threshold, and / or;

[0043] Among the gray value cross-entropies between the multiple second windows, the minimum cross-entropy is not less than a third set threshold.

[0044] Combined with any embodiment of the present disclosure, the step of enlarging the sizes of the multiple second windows is used for:

[0045] Enlarge the window sizes of the multiple second windows according to a set ratio, or;

[0046] Increase the window sizes of the multiple second windows according to a set size.

[0047] Combined with any embodiment of the present disclosure, the step of, in response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition, enlarging the multiple second windows until the set condition is satisfied, is used for:

[0048] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition and the number of iterations exceeding the iteration number threshold, determine the target window size according to the second window size obtained in the last iteration.

[0049] Combined with any embodiment of the present disclosure, the target window determination module, based on the occurrence frequencies of different pixel gray values among the multiple second windows, determines the target window size according to the second window size, and is used for:

[0050] In the case where the depth camera is a binocular depth camera, determine the second window size as the target window size;

[0051] In the case where the depth camera is a structured light depth camera, determine the target window size according to the window size of the second window and the number of pixels of the minimum target and the light spot determined by the structured light depth camera in the image.

[0052] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0053] A memory for storing the executable instructions executable by the processor;

[0054] A processor configured to execute the executable instructions in the memory to implement the steps of the method according to any one of the embodiments of the first aspect above.

[0055] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method according to any one of the embodiments of the first aspect above are implemented.

[0056] According to a fifth aspect of the embodiments of the present disclosure, a terminal device is provided, including the above-mentioned electronic device.

[0057] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0058] By adjusting multiple first window sizes of a depth camera to a reference window size to obtain multiple second windows, and based on the occurrence frequencies of different pixel gray values among the multiple second windows, determining a target window size according to the second window size and based on the target window size, performing feature matching processing on an image, it avoids setting the window size only relying on empirical values, and improves the setting efficiency of the window size and the accuracy of window matching.

[0059] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0061] Figure 1 is a flowchart of an image processing method shown according to an exemplary embodiment of the present disclosure;

[0062] Figure 2 is a schematic diagram of a window shown according to an exemplary embodiment of the present disclosure;

[0063] Figure 3 is another flowchart of an image processing method shown according to an exemplary embodiment of the present disclosure;

[0064] Figure 4 is a schematic diagram of a window adjustment device shown according to an exemplary embodiment of the present disclosure;

[0065] Figure 5 is a block diagram of an electronic device shown according to an exemplary embodiment of the present disclosure. Detailed Implementation Modes

[0066] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0067] The terms used in the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. The singular forms "a", "the", and "said" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0068] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0069] Figure 1 A flowchart of an image processing method according to an exemplary embodiment of the present disclosure is shown.

[0070] In step S101, according to the image captured by the depth camera, the first window sizes of a plurality of first windows of the depth camera are obtained, where the window is a rectangular area for determining the image feature matching range.

[0071] The depth camera includes any camera capable of obtaining image depth information, such as a binocular camera and a structured light camera.

[0072] When the depth camera is a binocular camera, within a first set distance, the left and right cameras can respectively obtain the left and right source images of the shooting target from different positions, and calculate to obtain the scene image; when the depth camera is a structured light camera, within a second set distance, the laser emitter can emit laser with a fixed structure to a parallel whiteboard, and the acquisition camera can obtain the source image and reference image of the shooting target, and calculate to obtain the scene image. The first set distance and the second set distance can be set as needed. After obtaining the scene image, a first window can be generated centered on the pixel at the center of the scene image, and multiple first windows with different sizes can be generated on both sides centered on this window, and the window sizes of the multiple first windows can be obtained respectively. In one example, it can be controlled that the center points of the generated multiple first windows are at the same horizontal position to save computing resources when adjusting the window size. For the specific window generation method, the present disclosure does not limit this.

[0073] In step S102, the first window sizes of the multiple first windows are adjusted to a reference window size to obtain multiple second windows, where the reference window size is determined from the first window sizes of the multiple first windows according to the search radius of the depth camera.

[0074] The search radius of the depth camera, that is, the range radius within which the depth camera can obtain image depth information, is a fixed parameter of the depth camera and can be obtained from the setting information of the depth camera. The multiple first windows after size unification are the multiple second windows.

[0075] In step S103, based on the occurrence frequencies of different pixel gray values among the multiple second windows, the target window size is determined according to the second window size.

[0076] The occurrence frequencies of the different pixel gray values can be the result determined by statistically counting all the pixels in the second window according to the magnitude and occurrence frequency of the gray levels. In one example, the occurrence frequencies of the different pixel gray values can be confirmed by a gray-scale statistical histogram, and the gray-scale statistical histogram can represent the number and occurrence frequency of a certain gray-scale pixel in the second window.

[0077] Based on the occurrence frequencies of different pixel gray values among the multiple second windows, the satisfaction of the set conditions can be used to determine the window size of the target window. In one example, the gray-scale relationship that satisfies the set conditions has both a certain difference and a certain consistency.

[0078] In step S104, based on the target window size, feature matching processing is performed on the image.

[0079] Within the obtained target window range, perform feature matching processing on the depth image obtained by the depth camera, that is, obtain the corresponding relationship of corresponding pixel points in different images obtained by the depth camera, and calculate the disparity map and depth map of the depth camera for subsequent imaging processes of the depth camera.

[0080] In the solution of the present disclosure, by adjusting the multiple first window sizes of the depth camera to a reference window size, multiple second windows are obtained, and based on the occurrence frequencies of different pixel gray values among the multiple second windows, the target window size is determined according to the second window size, improving the setting efficiency of the window size and the accuracy of window matching.

[0081] In an optional embodiment, the reference window size determined from the first window sizes of the multiple first windows according to the search radius of the depth camera includes:

[0082] Within the search radius of the depth camera, obtain the disparities of the multiple first windows respectively.

[0083] The disparity refers to the probability of pixel position difference in the first window and can be obtained through formula (1):

[0084]

[0085] where D is the disparity of the first window, and S ij is the difference of pixels within the first window, and the difference can be determined in the following manner:

[0086] Within the search radius of the depth camera, each of the first windows is determined by two different images. In the case where the depth camera is a binocular camera, Left ij and Right ij are the pixels at position (i, j) in the first windows determined by the left source image and the right source image respectively; in the case where the depth camera is a structured light camera, the Left ij and Right ij are the pixels at position (i, j) in the first windows determined by the source image and the reference image respectively. If the pixels of different images are the same under the same coordinates within the first window, the same result is recorded as 1, and if different, it is recorded as 0. The disparity D of the first window represents the probability of pixel difference in the first window determined by different images.

[0087] Obtain multiple first windows with disparities less than the first set threshold.

[0088] When the parallax is less than the first set threshold, it indicates that within the first window determined by different images, the pixel difference is less than the corresponding degree. The first set threshold is determined by the search radius of the depth camera and depends on the distance requirement during actual measurement. In one example, the first set threshold can be set to 50%.

[0089] In one example, the smallest window size among the multiple first windows can be determined as the reference window size.

[0090] The window with the smallest size among the multiple windows within the search radius with a parallax less than the first set threshold is determined as the reference window. Using the smallest size can reserve room for enlarging the window size in subsequent window size adjustments.

[0091] In the solution of the present disclosure, by determining the window with the smallest size among the multiple windows within the search radius with a difference between windows less than the first set threshold as the reference window, and adjusting the sizes of the multiple first windows of the depth camera to the reference window size to obtain multiple second windows, and based on the occurrence frequencies of different pixel gray values among the multiple second windows, determining the target window size according to the second window size, the setting efficiency of the window size and the accuracy of window matching are improved.

[0092] In an optional embodiment, the determining the target window size according to the second window size includes:

[0093] In response to the occurrence frequencies of different pixel gray values among the multiple second windows satisfying the set condition, determining the target window size according to the window size of the second window; in response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition, enlarging the multiple second windows until the set condition is satisfied.

[0094] The set condition can be set according to the statistical result of the occurrence frequencies of different pixel gray values among the multiple second windows. In the multiple second windows, each pixel has a gray level number for displaying an image. By statistically analyzing the gray level numbers, such as through a gray level statistical histogram and a gray frequency histogram, the occurrence frequency of a certain gray level in the second window can be determined. When the occurrence frequency of the gray value satisfies the set condition, the sizes of the multiple second windows do not need to be adjusted and can be output as the updated window size, and the target window size is determined according to this size. When the occurrence frequency of the gray value does not satisfy the set condition, the size of the second window can be adjusted by enlarging the second window and making a re-judgment, and through an iterative process until the set condition is satisfied, the window size of the adjusted second window is output as the updated size, and the target window size is determined according to this size.

[0095] In the solution described in the present disclosure, by determining the window with the smallest size among multiple windows within the search radius with a parallax less than a first set threshold as the reference window, and adjusting the sizes of multiple first windows of the depth camera to the reference window size to obtain multiple second windows, and based on whether the occurrence frequencies of different pixel gray values among the multiple second windows satisfy a set condition, correspondingly adjusting the window sizes of the multiple second windows, and determining the target window size according to the adjusted second window sizes, the setting efficiency of the window size and the accuracy of window matching are improved.

[0096] In an optional embodiment, that the occurrence frequencies of different pixel gray values among the multiple second windows satisfy the set condition includes: the degree of dispersion of the gray value information amounts of the multiple second windows is not greater than a second set threshold, and / or; among the gray value cross-entropies between the multiple second windows, the minimum cross-entropy is not less than a third set threshold.

[0097] The information amount is a measure of the amount of information. As Figure 2 shown, there are multiple second windows in the image. Taking the calculation of the information amount of window Wi as an example, first, it is necessary to obtain the gray-scale statistical histogram within window Wi, and obtain the gray-scale frequency histogram of window Wi, and calculate the information amount of window Wi through formula (2):

[0098]

[0099] where H i is the information amount of window W i , and P i is the frequency of the gray level i.

[0100] By the above method, calculate the information amounts of n second windows in the image respectively to obtain the set H = {H1 H2 … H n}.

[0101] In an example, the degree of dispersion of the information amount can be represented by the variance of the information amount, which can be achieved through formula (3):

[0102]

[0103] where Var(H) is the variance of the information amounts of the second windows in the image, which is used to represent the consistency between the second windows. When the degree of dispersion of the gray value information amounts of the multiple second windows is not greater than the second set threshold, the gray-scale statistical results of the multiple second windows have consistency.

[0104] Cross entropy is used to measure the difference information between two probability distributions. In the present disclosure, the difference between multiple second windows can be represented by cross entropy. Taking the cross entropy between window Wi and Wj as an example, first, the gray scale statistical histogram and the gray scale frequency histogram between the two windows need to be obtained respectively. And the cross entropy between window Wi and Wj is calculated by formula (4):

[0105]

[0106] where is the frequency of the gray level with the gray level number k in window W i ; is the frequency of the gray level with the gray level number k in window W j ;

[0107] By the above method, the cross entropy between any two of the n second windows in the image is calculated respectively to obtain a cross entropy matrix:

[0108]

[0109] When the minimum cross entropy Min(CrossH) in the matrix is not less than the third set threshold, the gray scale statistical results of the multiple second windows have differences.

[0110] It can be understood that the set condition can be a limiting condition that enables the multiple second windows to satisfy either the consistency or the difference property, or a limiting condition that enables the multiple second windows to satisfy both the consistency and the difference property simultaneously.

[0111] In the solution of the present disclosure, by adjusting the sizes of multiple first windows of the depth camera within the search radius to the reference window size to obtain multiple second windows, and based on whether the occurrence frequencies of different pixel gray values among the multiple second windows satisfy the consistency determined by the discrete degree of window information amount and the difference determined by the minimum cross entropy between windows, the window sizes of the multiple second windows are adjusted accordingly, and the target window size is determined according to the adjusted second window sizes, which improves the setting efficiency of the window size and the accuracy of window matching.

[0112] In an alternative embodiment, enlarging the sizes of the multiple second windows includes: enlarging the window sizes of the multiple second windows according to a set ratio, or; increasing the window sizes of the multiple second windows according to a set size.

[0113] For a second window that does not meet the set conditions, it can be enlarged in various ways to meet the set conditions. The enlargement can be either an equal-proportion enlargement of the window during each iteration according to a set ratio, or an increase of a fixed increment in the window size of the multiple second windows during each iteration according to a set size. In one example, the window size of the second window for each iteration can be increased by 10% of its original size to enlarge the window, as shown in formula (5):

[0114] Size update = Size update + ΔS, where ΔS = Size base × 10% (5)

[0115] Where, Size update is the window size of the updated second window. In the first iterative calculation, Size base is the window size of the base window. In the second and subsequent iterative calculations, Size base is the window size of the second window before this iterative update.

[0116] In the solution of the present disclosure, by determining the window with the smallest window size among multiple windows within the search radius whose window - to - window difference is less than the first set threshold as the reference window, and adjusting the window sizes of multiple first windows of the depth camera to the reference window size to obtain multiple second windows, and based on whether the occurrence frequencies of different pixel gray values among the multiple second windows meet the set conditions, the window sizes of the multiple second windows are enlarged through a preset method, and the target window size is determined according to the adjusted second window size, which improves the setting efficiency of the window size and the accuracy of window matching.

[0117] In an optional embodiment, the enlarging the multiple second windows until the set conditions are met in response to the occurrence frequencies of different pixel gray values among the multiple second windows not meeting the set conditions includes:[[]]

[0118] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not meeting the set conditions and the number of iterations exceeding the iteration number threshold, the target window size is determined according to the second window size obtained in the last iteration.

[0119] The iteration threshold is the number limit for the cyclic adjustment process that the multiple second windows can perform. In one example, to avoid continuously enlarging the multiple second windows, which may increase the computing pressure on the terminal device and reduce the computing efficiency of the processor in the terminal device during subsequent scene matching, the number of times of the cyclic adjustment process of the multiple second windows when the computing efficiency of the terminal device is lower than the set threshold can be set as the iteration threshold. In another example, to avoid continuously enlarging the multiple second windows, which may cause the multiple second windows to overlap, resulting in duplicate encoding during subsequent pixel encoding processing within the windows, the number of times of the cyclic adjustment process of the multiple second windows when duplicate encoding occurs during pixel encoding processing within the windows of the terminal device can be set as the iteration threshold. It can be understood that the iteration threshold can be determined separately according to the methods described in the above two examples, or can be determined by combining the above two examples. For example, when the computing efficiency of the terminal device is lower than the set threshold and duplicate encoding occurs during pixel encoding processing within the windows of the terminal device, the number of times of the cyclic adjustment process of the multiple second windows can be set as the iteration threshold.

[0120] In the solution described in the present disclosure, by adjusting the sizes of multiple first windows of the depth camera to the reference window size, multiple second windows are obtained, and in response to the occurrence frequencies of different pixel gray values among the multiple second windows not meeting the set conditions and the iteration times exceeding the iteration threshold, the target window size is determined according to the second window size obtained in the last iteration, and the target window size is determined according to the second window size, thereby improving the setting efficiency of the window size and the accuracy of window matching.

[0121] In an optional embodiment, determining the target window size according to the second window size includes: when the depth camera is a binocular camera, determining the second window size as the target window size.

[0122] Since in a binocular camera, by performing disparity calculation on two source images obtained by the left camera and the right camera to measure the distance of the range captured by the image, a test sample image can be directly obtained. The influencing factors of the window size mainly depend on the window information amount and the difference between windows. Therefore, the window size of the second window that meets the preset conditions can be directly determined as the target window size.

[0123] When the depth camera is a structured light depth camera, the target window size is determined according to the window size of the second window and the number of pixels of the minimum target and the light spot determined by the structured light depth camera in the image.

[0124] In a structured light camera, a test pattern is collected jointly by a laser emitter and an acquisition camera. Since the laser emitter can project light with certain structural features onto the object to be photographed, different-sized light spots will be generated due to different depth regions of the target object. Therefore, the influencing factors of the window size include not only the window information amount and the difference between windows, but also the minimum target size to be photographed and the spot size of the light spot. In one example, the window size of the target window can be obtained by formula (6):

[0125]

[0126] where Size object is the window size of the target window, Size new is the window size of the second window that meets the preset conditions. S1 is the number of pixels of the smallest target image defined by the application scenario in the image. The smallest target is a scene area with an obvious depth difference from the surrounding environment. The difference threshold for judging the depth difference between the scene area and the depth of the surrounding environment can be set according to different application scenarios. S2 is the number of pixels of the light spot in the image.

[0127] In the solution of the present disclosure, by adjusting the first window sizes of a depth camera to a reference window size, a plurality of second windows are obtained, and based on the occurrence frequencies of different pixel gray values among the plurality of second windows, the window size of the target window is determined respectively according to different types of cameras, which improves the setting efficiency of the window size and the accuracy of window matching.

[0128] Figure 3 Another flowchart of an image processing method according to an exemplary embodiment of the present disclosure is shown.

[0129] In step S303, adjust the first window sizes of the multiple first windows to a reference window size to obtain multiple second windows: Among the multiple first windows determined by the depth camera, determine the reference window size and uniformly adjust the first window sizes of the multiple first windows to the reference window size. In step S304, within the search radius, calculate the information amount of the multiple second windows: Obtain the gray-scale statistical histogram within the multiple second windows and obtain the gray-scale frequency histogram of the windows to calculate the information amount of each second window. In step S305, when the information dispersion degree of the multiple second windows is not greater than the second set threshold, determine that the window size of the updated window is equal to the window size of the second window; when the information amount of the multiple second windows is greater than the second set threshold, determine whether the number of iterations exceeds the iteration number threshold. In step S306, determine the window size of the updated window as the window size of the second window. In step S307, in the gray-scale value cross-entropy among the multiple second windows, when the minimum cross-entropy is not less than the third set threshold, output the window size of the updated window, that is, the window size of the second window; when the minimum cross-entropy among the cross-entropies between the multiple second windows is less than the third set threshold, determine whether the number of iterations exceeds the iteration number threshold. In step S309, when the number of iterations does not exceed the iteration number threshold, in step S310, enlarge the window size of the second window to obtain the window size of the updated window; when the number of iterations exceeds the iteration number threshold, in step S311, output the window size of the updated window, that is, the window size of the second window enlarged during the previous iteration process.

[0130] In the solution of the present disclosure, by determining the window with the smallest window size among multiple windows within the search radius where the difference between windows is less than the first set threshold as the reference window, and adjusting the multiple first window sizes of the depth camera to the reference window size to obtain multiple second windows, and performing an enlargement process on the window sizes of the multiple second windows based on whether the occurrence frequencies of different pixel gray-scale values among the multiple second windows meet the set conditions, and determining the target window size according to the adjusted second window size, the setting efficiency of the window size and the accuracy of window matching are improved.

[0131] In one example, the distance limitation of the sample scene image captured by the depth camera can also be extended. Specifically, the window sizes of a group of the first windows can be measured at intervals of a certain distance, and then the weighted average of the corresponding first windows in each group can be calculated to expand the shooting distance of the depth camera for the current application scenario and reduce the limitation of the shooting distance of the depth camera head during the window adjustment process.

[0132] For each of the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should understand that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously.

[0133] Secondly, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0134] Corresponding to the foregoing method embodiments for implementing application functions, the present disclosure also provides embodiments of an apparatus for implementing application functions and corresponding terminals.

[0135] A block diagram of a window adjustment apparatus shown in an exemplary embodiment of the present disclosure is as Figure 4 shown, and the apparatus includes:

[0136] A first window acquisition module 401: configured to acquire the first window sizes of a plurality of first windows of the depth camera according to an image captured by the depth camera, where the window is a rectangular area for determining an image feature matching range;

[0137] A second window acquisition module 402: configured to adjust the first window sizes of the plurality of first windows to a reference window size to obtain a plurality of second windows, where the reference window size is determined from the first window sizes of the plurality of first windows according to a search radius of the depth camera;

[0138] A target window determination module 403: configured to determine a target window size according to the second window size based on the occurrence frequencies of different pixel gray values among the plurality of second windows.

[0139] In combination with any implementation manner of the present disclosure, in the second window acquisition module, the reference window size determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera is used for:

[0140] Respectively acquire the disparities of the plurality of first windows within the search radius of the depth camera;

[0141] Acquire a plurality of first windows with disparities less than a first set threshold;

[0142] Determine the smallest window size among the plurality of first windows as the reference window size.

[0143] In combination with any implementation manner of the present disclosure, the second window acquisition module is configured to determine a target window size according to the second window size, and specifically configured to:

[0144] In response to the occurrence frequencies of different pixel gray values among the multiple second windows satisfying a set condition, determine a target window size according to the window size of the second window;

[0145] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition, enlarge the multiple second windows until the set condition is satisfied.

[0146] Combined with any implementation manner of the present disclosure, the occurrence frequencies of different pixel gray values among the multiple second windows satisfying the set condition are used for:

[0147] The degree of dispersion of the gray value information amount of the multiple second windows is not greater than a second set threshold, and / or;

[0148] Among the gray value cross-entropies between the multiple second windows, the minimum cross-entropy is not less than a third set threshold.

[0149] Combined with any implementation manner of the present disclosure, the enlarging the size of the multiple second windows is used for:

[0150] Enlarge the window size of the multiple second windows according to a set ratio, or;

[0151] Increase the window size of the multiple second windows according to a set size.

[0152] Combined with any implementation manner of the present disclosure, the responding to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition and enlarging the multiple second windows until the set condition is satisfied is used for:

[0153] In response to the occurrence frequencies of different pixel gray values among the multiple second windows not satisfying the set condition and the number of iterations exceeding the iteration number threshold, determine the target window size according to the second window size obtained in the last iteration.

[0154] Combined with any implementation manner of the present disclosure, the target window determination module determines the target window size according to the window size of the second window based on the occurrence frequencies of different pixel gray values among the multiple second windows, and is used for:

[0155] In the case that the depth camera is a binocular depth camera, determine the second window size as the target window size;

[0156] In the case that the depth camera is a structured light depth camera, determine the target window size according to the window size of the second window and the number of pixels of the minimum target and the light spot determined by the structured light depth camera in the image.

[0157] For the apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present disclosure. A person of ordinary skill in the art can understand and implement it without creative work.

[0158] Figure 5 FIG. shows a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0159] Please refer to the appendix Figure 5 , which exemplarily shows a block diagram of an electronic device. For example, the apparatus 500 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0160] Referring to Figure 5 , the apparatus 500 may include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, a multimedia component 508, an audio component 510, an input / output (I / O) interface 512, a sensor component 514, and a communication component 516.

[0161] The processing component 502 generally controls the overall operation of the apparatus 500, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0162] The memory 504 is configured to store various types of data to support the operation of the device 500. Examples of such data include instructions for any application or method operating on the device 500, contact data, phone book data, messages, pictures, videos, and the like. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0163] The power component 506 provides power to the various components of the device 500. The power component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 500.

[0164] The multimedia component 508 includes a screen that provides an output interface between the device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0165] The audio component 510 is configured to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC) that is configured to receive external audio signals when the device 500 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 504 or transmitted via the communication component 516. In some embodiments, the audio component 510 includes a speaker for outputting audio signals.

[0166] The I / O interface 512 provides an interface between the processing component 502 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0167] The sensor component 514 includes one or more sensors for providing a status assessment of various aspects of the device 500. For example, the sensor component 514 can detect the on / off state of the device 500, the relative positioning of components, such as the display and keypad of the device 500. The sensor component 514 can detect a change in the position of the device 500 or a component of the device 500, the presence or absence of user contact with the device 500, the orientation or acceleration / deceleration of the device 500, and the temperature change of the device 500. The sensor component 514 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 514 can include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 514 can include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0168] The communication component 516 is configured to facilitate communication between the device 500 and other devices in a wired or wireless manner. The device 500 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0169] In an exemplary embodiment, the device 500 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the power supply method of the above electronic device.

[0170] In an exemplary embodiment of the present disclosure, there is provided a non-transitory computer-readable storage medium including instructions, such as a memory 504 including instructions, which can be executed by a processor 520 of the device 500 to complete the power supply method of the above electronic device. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0171] Other embodiments of the present disclosure will be readily apparent to those skilled in the art in view of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only illustrative, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0172] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtaining the first window sizes of a plurality of first windows of the depth camera according to an image captured by the depth camera, where the window is an area for determining an image feature matching range; Adjusting the first window sizes of the plurality of first windows to a reference window size to obtain a plurality of second windows, where the reference window size is determined from the first window sizes of the plurality of first windows according to a search radius of the depth camera; Determining a target window size according to the second window size based on the occurrence frequencies of different pixel gray values among the plurality of second windows; Performing feature matching processing on the image based on the target window size.

2. The method according to claim 1, wherein The reference window size determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera includes: Respectively obtaining the disparities of the plurality of first windows within the search radius of the depth camera; Obtaining the plurality of first windows with disparities less than a first set threshold; Determining the smallest window size among the plurality of first windows as the reference window size.

3. The method according to claim 1, wherein The determining the target window size according to the second window size includes: In response to the occurrence frequencies of different pixel gray values among the plurality of second windows satisfying a set condition, determining the target window size according to the window size of the second window; In response to the occurrence frequencies of different pixel gray values among the plurality of second windows not satisfying the set condition, enlarging the plurality of second windows until the set condition is satisfied.

4. The method according to claim 3, characterized in that, The occurrence frequencies of different pixel gray values among the plurality of second windows satisfying the set condition includes: The degree of dispersion of the gray value information amounts of the plurality of second windows is not greater than a second set threshold, and / or; Among the gray value cross-entropies between the plurality of second windows, the minimum cross-entropy is not less than a third set threshold.

5. The method according to claim 3, characterized in that, The enlarging the plurality of second windows includes: Enlarging the window sizes of the plurality of second windows according to a set ratio, or; Increasing the window sizes of the plurality of second windows according to a set size.

6. The method according to claim 3, characterized in that The in response to the occurrence frequencies of different pixel gray values among the plurality of second windows not satisfying the set condition, enlarging the plurality of second windows until the set condition is satisfied includes: In response to the occurrence frequencies of different pixel gray values among the plurality of second windows not satisfying the set condition and the number of iterations reaching an iteration number threshold, determining the target window size according to the second window size obtained in the last iteration.

7. The method according to claim 3, wherein The determining the target window size according to the second window size includes: In the case where the depth camera is a binocular depth camera, determining the second window size as the target window size; In the case where the depth camera is a structured light depth camera, determining the target window size according to the window size of the second window and the number of pixels of the smallest target and the light spot determined by the structured light depth camera in the image.

8. A window adjustment device, characterized in that, The apparatus includes: A first window obtaining module: obtaining the first window sizes of a plurality of first windows of the depth camera according to an image captured by the depth camera, where the window is a rectangular area for determining an image feature matching range; Second window acquisition module: configured to adjust the first window sizes of the plurality of first windows to a reference window size, obtaining a plurality of second windows, wherein the reference window size is determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera; Target window determination module: configured to determine a target window size according to the second window size based on the occurrence frequencies of different pixel gray values among the plurality of second windows; Based on the target window size, perform feature matching processing on the image.

9. The device according to claim 8, characterized in that, In the second window acquisition module, the reference window size determined from the first window sizes of the plurality of first windows according to the search radius of the depth camera is used for: Within the search radius of the depth camera, respectively obtain the disparities of the plurality of first windows; Obtain the plurality of first windows whose disparities are less than a first set threshold; Determine the smallest window size among the plurality of first windows as the reference window size.

10. The device according to claim 8, characterized in that, The second window acquisition module is configured to determine a target window size according to the second window size, specifically configured to: In response to the occurrence frequencies of different pixel gray values among the plurality of second windows satisfying a set condition, determine the target window size according to the window size of the second window; In response to the occurrence frequencies of different pixel gray values among the plurality of second windows not satisfying the set condition, enlarge the plurality of second windows until the set condition is satisfied.

11. The device according to claim 10, characterized in that, The occurrence frequencies of different pixel gray values among the plurality of second windows satisfying the set condition are used for: The degree of dispersion of the gray value information amounts of the plurality of second windows is not greater than a second set threshold, and / or; Among the gray value cross-entropies between the plurality of second windows, the minimum cross-entropy is not less than a third set threshold.

12. The device according to claim 10, characterized in that, The enlarging the sizes of the plurality of second windows is used for: Enlarge the window sizes of the plurality of second windows according to a set ratio, or; Increase the window sizes of the plurality of second windows according to a set size.

13. The device according to claim 10, characterized in that, The "in response to the occurrence frequencies of different pixel gray values among the plurality of second windows not satisfying the set condition, enlarge the plurality of second windows until the set condition is satisfied" is used for: In response to the occurrence frequencies of different pixel gray values among the plurality of second windows not satisfying the set condition and the number of iterations exceeding the iteration number threshold, determine the target window size according to the second window size obtained in the last iteration.

14. The device according to claim 10, wherein, The target window determination module, based on the occurrence frequencies of different pixel gray values among the plurality of second windows, determines the target window size according to the second window size, and is used for: In the case where the depth camera is a binocular depth camera, determine the second window size as the target window size; In the case where the depth camera is a structured light depth camera, determine the target window size according to the window size of the second window and the number of pixels of the smallest target and the light spot determined by the structured light depth camera in the image.

15. An electronic device, characterized in that, The electronic device includes: A memory, configured to store processor-executable instructions; A processor, configured to execute the executable instructions in the memory to implement the steps of the method according to any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the described program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

17. A terminal device, characterized in that, It includes the electronic device according to claim 15.

Images