Image depth determination method, device, electronic device and storage medium

Abstract

The present application provides an image depth determination method, device, electronic device and storage medium, which relates to the field of image processing technology. The method includes: acquiring multiple frames of binocular images, and obtaining a depth estimation value of a target frame image based on a preset depth estimation algorithm; wherein the target frame image is an image of any frame in the multiple frames of binocular images; determining the disparity range of each pixel in the target frame image based on the depth estimation value; performing iterative calculations within the disparity range according to a binocular matching algorithm to determine the disparity of each pixel to complete image depth estimation. The image depth determination method provided by the embodiment of the present application can reduce the amount of calculation for determining image depth, save computing resources, and improve the efficiency of image depth determination.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device, electronic device and storage medium for determining image depth. Background Art

[0002] Image depth refers to the number of bits actually required to store the grayscale or color of the image in the pixel depth. After determining the image depth, it can be used to achieve background blur, locate the distance of objects in the image, and calculate the current camera's posture. Currently, binocular cameras are mainly used to measure image depth. The principle of binocular depth estimation is based on the left eye, finds the matching points corresponding to the points on the left image in the right image, and then obtains the corresponding depth based on the principle of triangulation. In this case, it is necessary to determine the matching cost of each point between the maximum and minimum values ​​of the image disparity. Therefore, there are problems of large amount of calculation and low efficiency of depth calculation. Summary of the invention

[0003] Based on this, the purpose of the embodiments of the present application is to provide a method, device, electronic device and storage medium for determining image depth, which calculates the motion information between adjacent frames to determine the disparity estimate of each point in the binocular image, and then uses the disparity estimate as the initial value for iterative calculation by the binocular matching algorithm to determine the depth value of each point in the binocular image, thereby reducing the amount of calculation for determining the image depth, saving computing resources and improving the efficiency of image depth determination.

[0004] In a first aspect, an embodiment of the present application provides a method for determining image depth, comprising:

[0005] Acquire multiple frames of binocular images, and obtain a depth estimation value of a target frame image based on a preset depth estimation algorithm; wherein the target frame image is an image of any frame in the multiple frames of binocular images;

[0006] Determine the disparity range of each pixel in the target frame image based on the depth estimation value;

[0007] An iterative calculation is performed within the disparity range according to the binocular matching algorithm to determine the disparity of each pixel to complete the image depth estimation.

[0008] In the above implementation process, a rough depth range can be determined for each point, and the specific depth value can be directly determined in the determined disparity range in the subsequent calculation process. Moreover, since this initial depth value is not required to be accurate and only acts to provide the disparity range, such calculation amount will not occupy the computing resources of the binocular matching algorithm. The initial depth estimation value obtained by calculation can realize the disparity range of different orders at different positions of the original image with the same disparity calculation, thereby reducing the amount of calculation and the amount of storage occupied by the calculation.

[0009] Optionally, before obtaining the depth estimation value of the target frame image based on a preset depth estimation algorithm, the method may further include:

[0010] Performing resolution reduction processing on the multiple frames of binocular images to obtain bypass images corresponding to the multiple frames of binocular images respectively;

[0011] The bypass image is used as input data of the preset depth estimation algorithm to obtain the depth estimation value based on the preset depth estimation algorithm.

[0012] In the above implementation process, since the accuracy of disparity estimation is not required, lower resolution images can be used for calculation, which can reduce the computing resources occupied by disparity estimation and avoid determining the corresponding disparity value for each pixel from the range of maximum disparity value and minimum disparity value, which can greatly reduce the amount of calculation and save computing resources.

[0013] Optionally, the preset depth estimation algorithm may include: an optical flow algorithm, an adjacent frame difference method or a three-frame difference method; and obtaining a depth estimation value of the target frame image based on the preset depth estimation algorithm includes:

[0014] The motion state between adjacent frames in a plurality of bypass images is calculated based on an optical flow algorithm, an adjacent frame difference method or a three-frame difference method to obtain a depth estimation value of the target frame image; wherein the bypass image includes the plurality of binocular images, or the plurality of binocular images, after the resolution of the plurality of binocular images is reduced, and the images respectively correspond to the plurality of binocular images.

[0015] Optionally, before acquiring multiple frames of binocular images and obtaining a depth estimation value of a target frame image based on a preset depth estimation algorithm, the method may include:

[0016] The multiple frames of binocular images are encoded based on statistical transformation, the pixel grayscale value of the image in the neighborhood window is compared with the pixel grayscale value of the window center, a comparison result is obtained, and the comparison result is mapped into a bit string, and the value of the bit string is used as the statistical transformation value of the center pixel.

[0017] In the above implementation process, the position characteristics of the pixels in the window can be retained through the implementation of statistical transformation. Since the statistical transformation is not sensitive to the overall brightness changes of the image, when comparing the relative grayscale relationship, even if the brightness of the left and right images is inconsistent, a good matching effect can be obtained. Therefore, it has high robustness to brightness deviation and can reduce false matching caused by lighting differences.

[0018] Optionally, determining the disparity range of each pixel in the target frame image based on the depth estimation value may include:

[0019] Calculating a matching cost based on the disparity range;

[0020] Cost aggregation is performed on the matching costs, and matching points corresponding to each position are selected from the binocular image to determine the disparity.

[0021] Optionally, the calculating the matching cost based on the disparity range includes:

[0022] Based on the encoded results of the multiple frames of binocular images, the Hamming distance between two points in the left and right views of the binocular images is calculated to determine the matching cost.

[0023] In summary, the image depth determination method provided in the embodiment of the present application is based on the continuity of parallax changes. When determining the depth, the estimated depth range is used to pre-estimate the local parallax of the points on the image, and then a smaller parallax error interval is assumed on the basis of the pre-estimation for calculation and matching. The parallax estimate value of each point in the binocular image can be determined based on the calculation of motion information between adjacent frames, and then the binocular matching algorithm uses the parallax estimate value as the initial value for iterative calculation to determine the depth value of each point in the binocular image. This can reduce the amount of calculation for determining the image depth, save computing resources, and improve the efficiency of image depth determination.

[0024] In a second aspect, an embodiment of the present application provides an image depth determination device, which may include:

[0025] The estimation module is used to acquire multiple frames of binocular images and obtain the depth estimation value of the target frame image based on a preset depth estimation algorithm.

[0026] The target frame image is an image of any frame in the multiple frames of binocular images.

[0027] The disparity range calculation module is used to determine the disparity range of each pixel in the target frame image based on the depth estimation value.

[0028] The disparity determination module is used to perform iterative calculations within the disparity range according to a binocular matching algorithm to determine the disparity of each pixel point to complete image depth estimation.

[0029] Optionally, the image depth determination device may also include a preprocessing module, which is used to reduce the resolution of the multiple frames of binocular images to obtain bypass images corresponding to the multiple frames of binocular images respectively; and use the bypass images as input data of the preset depth estimation algorithm to obtain the depth estimation value based on the preset depth estimation algorithm.

[0030] Optionally, the preset depth estimation algorithm may include: an optical flow algorithm, an adjacent frame difference method or a three-frame difference method; the estimation module may be specifically used to: calculate the motion state between adjacent frames in a plurality of bypass images based on the optical flow algorithm, the adjacent frame difference method or the three-frame difference method to obtain a depth estimation value of the target frame image; wherein the bypass image includes the plurality of binocular images, or the plurality of binocular images, after the resolution is reduced, images corresponding to the plurality of binocular images.

[0031] Optionally, the image depth determination device may also include an encoding module for encoding the multiple frames of binocular images based on statistical transformation, comparing the pixel grayscale value of the image in the neighborhood window with the pixel grayscale value of the window center to obtain a comparison result, and mapping the comparison result to a bit string, using the value of the bit string as the statistical transformation value of the center pixel.

[0032] Optionally, the disparity range calculation module may be used to:

[0033] Calculating a matching cost based on the disparity range;

[0034] Cost aggregation is performed on the matching costs, and matching points corresponding to each position are selected from the binocular image to determine the disparity.

[0035] Furthermore, the parallax range calculation module can be specifically used for:

[0036] Based on the encoded results of the multiple frames of binocular images, the Hamming distance between two points in the left and right views of the binocular images is calculated to determine the matching cost.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above-mentioned implementation methods are executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1A schematic diagram of the steps of the method for determining image depth provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the image depth determination effect provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the steps of determining the disparity of each pixel in a binocular image provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of an image depth determination device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented by a dedicated hardware-based system that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0045] During the research, the applicant discovered that in the current method of estimating depth from an image, the process of finding a match generally assumes that the left and right cameras have completed epipolar correction, that is, the vertical y coordinates corresponding to the matching point pairs are the same, and only the best match is found in the horizontal x direction. The algorithm assumes the maximum and minimum disparity of the image, and calculates the matching costs corresponding to each level of disparity in the interval between the minimum and maximum disparity at each point in the image, and then uses the algorithm to find the disparity corresponding to the minimum matching cost. The larger the interval between the maximum and minimum disparity, the more storage and computation required by the algorithm. Therefore, there are problems of large computational complexity and low depth calculation efficiency.

[0046] Based on this, an embodiment of the present application provides a method for determining image depth, which determines the estimated disparity value of each point in the binocular image based on the motion information between adjacent frames of the binocular image, and then uses the estimated disparity value as the initial value for iterative calculation by the binocular matching algorithm to determine the depth value of each point in the binocular image. This can reduce the amount of calculation for determining the image depth, save computing resources, and improve the efficiency of image depth determination.

[0047] Please see Figure 1 , Figure 1 This is a schematic diagram of the steps of the method for determining image depth provided in an embodiment of the present application. The method for determining image depth may include the following steps:

[0048] In step S11, multiple frames of binocular images are acquired, and depth estimation values ​​of target frame images are obtained based on a preset depth estimation algorithm.

[0049] The target frame image is an image of any frame in the multiple frames of binocular images.

[0050] The applicant has found that based on the continuity of disparity changes, the disparity of the same position in adjacent frames of the binocular image will not change dramatically. Therefore, an algorithm capable of estimating the motion of two frames of images can be used to calculate the motion state of the two frames of images from time t to time t+1 or from time t-1 to time t, and estimate the disparity through the motion direction and speed of the image pixels, thereby providing an initial depth estimate for the binocular estimation algorithm. Among them, the target frame image can be understood as the image at time t.

[0051] In step S12, the disparity range of each pixel in the target frame image is determined based on the depth estimation value.

[0052] In step S13, an iterative calculation is performed within the disparity range according to a binocular matching algorithm to determine the disparity of each pixel point to complete image depth estimation.

[0053] Among them, the binocular matching algorithm used in the embodiment of the present application takes the SGBM (Semi-Global Block Matching) algorithm as an example. In actual application scenarios, SGM (Semi-Global Matching) or other algorithms capable of binocular matching may also be selected.

[0054] For example, see Figure 2 , Figure 2 The schematic diagram of the image depth determination effect provided in the embodiment of the present application, it can be understood that when the depth is determined by binocular matching at present, the depth determination range for each point in the image is certain, such as Figure 2The range shown in , and through the method adopted in the embodiment of the present application, a rough depth range can be determined for each point respectively, such as the disparity range of the close point A can be further determined as [30-25, 30+25], the disparity range of the medium-distance point B can be further determined as [50-25, 50+25], and the disparity range of the far-distance point C can be further determined as [60-25, 60+25]. In the subsequent calculation process, the specific depth value is directly determined in the determined disparity range, thereby reducing the amount of calculation when calculating the depth of binocular matching, thereby improving the calculation efficiency.

[0055] In addition, in the embodiment provided by the present application, since the initial depth value is not required to be accurate and is only used to provide a disparity range, such a calculation amount will not occupy the computing resources of the binocular matching algorithm, so the calculation amount can be directly calculated on the CPU without any hardware acceleration. The initial depth estimate value obtained by calculation can realize the disparity range of different orders at different positions of the original image with the same disparity calculation, thereby reducing the amount of calculation and the amount of storage occupied by the calculation.

[0056] In an optional embodiment, before step S11, the method provided in the embodiment of the present application may also include reducing the resolution of the multiple frames of binocular images to obtain bypass images corresponding to the multiple frames of binocular images respectively; and using the bypass images as input data of the preset depth estimation algorithm to obtain the depth estimation value based on the preset depth estimation algorithm.

[0057] Exemplarily, the present application illustrates the steps of depth determination using a main path and a bypass path. In the main path, the existing SGBM algorithm can be used to calculate the depth value, wherein the processed data is the original binocular image. In the bypass path, a simplified binocular estimation algorithm or other algorithms such as optical flow that estimate the motion between two frames can be used to estimate the disparity of the image. Since the accuracy of the disparity estimation is not required, a lower resolution image can be used for calculation, and the lower resolution image can be obtained by reducing the resolution of the original image.

[0058] Assume that the calculation frequency of the main SGBM algorithm can be set to 30fps, while the bypass can be set to only 5fps or lower, so that a smaller amount of calculation can be used to obtain a depth estimate. Based on the depth estimate, the disparity range of each pixel in the original image is set, so that the SGBM algorithm in the main path can determine the accurate disparity value based on the disparity range, avoiding the need to determine the corresponding disparity value for each pixel from the range of maximum disparity value and minimum disparity value, which can greatly reduce the amount of calculation and save computing resources.

[0059] In an optional implementation, the preset depth estimation algorithm may include: an optical flow algorithm, an adjacent frame difference method or a three-frame difference method; the step of determining the depth estimation in step S11 may specifically include: calculating the motion state between adjacent frames in multiple frames of bypass images based on the optical flow algorithm, the adjacent frame difference method or the three-frame difference method to obtain the depth estimation value of the target frame image.

[0060] The bypass image includes the multiple frames of binocular images, or images corresponding to the multiple frames of binocular images after the multiple frames of binocular images are processed to reduce the resolution.

[0061] In an optional embodiment, in an actual scene, due to the inconsistent light intensity received by the left and right cameras at different viewing angles in the binocular camera, the possible differences in camera gain and level, and the different noises in different channels of image acquisition, brightness differences may be caused. In order to reduce the adverse effects caused by the above-mentioned brightness differences, before step S11, the method provided in the embodiment of the present application may also include a step of encoding the binocular image, and the matching error is calculated by encoding the image. Specifically, the encoding step may include: encoding the multiple frames of binocular images based on a statistical transformation, comparing the pixel grayscale value of the image in the neighborhood window with the pixel grayscale value of the window center to obtain a comparison result, and mapping the comparison result to a bit string, and using the value of the bit string as the statistical transformation value of the center pixel.

[0062] Exemplarily, the window size of the domain window can be n×m, where n and m are both odd numbers. The grayscale values ​​of the pixels in the window are compared with the grayscale values ​​of the center pixels of the window, and the Boolean values ​​obtained by the comparison are mapped to a bit string. Finally, the value of the bit string is used as the statistical (Census) transformation value of the center pixel.

[0063] It can be seen that the embodiment of the present application can retain the position characteristics of the pixels in the window through statistical transformation. Since the statistical transformation is not sensitive to the overall brightness changes of the image, when comparing the relative grayscale relationship, even if the brightness of the left and right images is inconsistent, a good matching effect can be obtained. Therefore, it has high robustness to brightness deviation and can reduce mismatching caused by lighting differences.

[0064] In an optional embodiment, for step S12, the present application embodiment provides a method for determining the parallax of each pixel in the binocular image. Figure 3 , Figure 3 A schematic diagram of the steps of determining the disparity of each pixel in a binocular image provided in an embodiment of the present application, wherein the steps of determining the disparity of each pixel in a binocular image may include:

[0065] In step S31 , a matching cost is calculated based on the disparity range.

[0066] Specifically, since the binocular image has undergone a statistical transformation and encoding step, the matching cost can be calculated by directly calculating the Hamming distance between two points in the binocular image, such as comparing the grayscale value of a pixel in a domain window of size 3×3 with the grayscale value of the pixel in the center of the window:

[0067]

[0068]

[0069] The two cencus sequences are XORed to obtain {01110010}, resulting in a Hamming distance of 4.

[0070] Among them, each point on the image will be calculated in the corresponding disparity range, so the size of the calculation result is h*w*d, where d represents the disparity range, h is the image height, and w is the image width.

[0071] In step S32, the matching costs are aggregated, and matching points corresponding to each position are selected from the binocular image to determine the disparity.

[0072] Exemplarily, the method of performing cost aggregation can refer to the path aggregation method in the SGBM algorithm, and the disparity value of the current point pixel is optimized by the disparity value of the surrounding pixel points, which is not specifically introduced in the embodiments of the present application.

[0073] In summary, the image depth determination method provided in the embodiment of the present application is based on the continuity of parallax changes. When determining the depth, the estimated depth range is used to pre-estimate the local parallax of the points on the image, and then a smaller parallax error interval is assumed on the basis of the pre-estimation for calculation and matching. The parallax estimate value of each point in the binocular image can be determined based on the calculation of motion information between adjacent frames, and then the binocular matching algorithm uses the parallax estimate value as the initial value for iterative calculation to determine the depth value of each point in the binocular image. This can reduce the amount of calculation for determining the image depth, save computing resources, and improve the efficiency of image depth determination.

[0074] Based on the same inventive concept, the present application embodiment also provides an image depth determination device 40, please refer to Figure 4 , Figure 4 This is a schematic diagram of an image depth determination device provided in an embodiment of the present application. The image depth determination device 40 may include:

[0075] The estimation module 41 is used to acquire multiple frames of binocular images and obtain a depth estimation value of a target frame image based on a preset depth estimation algorithm.

[0076] The target frame image is an image of any frame in the multiple frames of binocular images.

[0077] The disparity range calculation module 42 is used to determine the disparity range of each pixel in the target frame image based on the depth estimation value.

[0078] The disparity determination module 43 is used to perform iterative calculations within the disparity range according to a binocular matching algorithm to determine the disparity of each pixel point to complete image depth estimation.

[0079] It can be seen that the embodiment of the present application can determine a rough depth range for each point respectively, and directly determine the specific depth value in the determined disparity range in the subsequent calculation process, and since this initial depth value is not required to be accurate, it only acts to provide the disparity range, so such calculation amount will not occupy the computing resources of the binocular matching algorithm. The initial depth estimate value obtained by calculation can realize the disparity range of different orders at different positions of the original image with the same disparity calculation, thereby reducing the amount of calculation and the amount of storage occupied by the calculation.

[0080] Optionally, the image depth determination device 40 may further include a preprocessing module, which is used to reduce the resolution of the multiple frames of binocular images to obtain bypass images corresponding to the multiple frames of binocular images respectively; and use the bypass images as input data of the preset depth estimation algorithm to obtain the depth estimation value based on the preset depth estimation algorithm.

[0081] It can be seen that in the embodiment of the present application, since the accuracy of the disparity estimation is not required, a lower resolution image can be used for calculation, which can reduce the computing resources occupied by obtaining the disparity estimation and avoid determining the corresponding disparity value for each pixel from the range of maximum disparity value and minimum disparity value, which can greatly reduce the amount of calculation and save computing resources.

[0082] Optionally, the estimation module 41 can be specifically used to: calculate the motion state between adjacent frames in multiple frames of bypass images based on an optical flow algorithm, an adjacent frame difference method or a three-frame difference method to obtain a depth estimation value of the target frame image; wherein the bypass image includes the multiple frames of binocular images, or the multiple frames of binocular images, after the resolution is reduced, and the images corresponding to the multiple frames of binocular images are respectively processed.

[0083] Optionally, the image depth determination device 40 may also include an encoding module for encoding the multiple frames of binocular images based on statistical transformation, comparing the pixel grayscale value of the image in the neighborhood window with the pixel grayscale value of the window center to obtain a comparison result, and mapping the comparison result to a bit string, using the value of the bit string as the statistical transformation value of the center pixel.

[0084] It can be seen that the embodiment of the present application can retain the position characteristics of the pixels in the window through the implementation of statistical transformation. Since the statistical transformation is not sensitive to the overall brightness changes of the image, when comparing the relative grayscale relationship, even if the brightness of the left and right images is inconsistent, a good matching effect can be obtained. Therefore, it has high robustness to brightness deviation and can reduce mismatching caused by lighting differences.

[0085] Optionally, the parallax range calculation module 42 may be used to:

[0086] Calculating a matching cost based on the disparity range;

[0087] Cost aggregation is performed on the matching costs, and matching points corresponding to each position are selected from the binocular image to determine the disparity.

[0088] Furthermore, the parallax range calculation module 42 may be specifically used for:

[0089] Based on the encoded results of the multiple frames of binocular images, the Hamming distance between two points in the left and right views of the binocular images is calculated to determine the matching cost.

[0090] In summary, the image depth determination method provided in the embodiment of the present application is based on the continuity of parallax changes. When determining the depth, the estimated depth range is used to pre-estimate the local parallax of the points on the image, and then a smaller parallax error interval is assumed on the basis of the pre-estimation for calculation and matching. The parallax estimate value of each point in the binocular image can be determined based on the calculation of motion information between adjacent frames, and then the binocular matching algorithm uses the parallax estimate value as the initial value for iterative calculation to determine the depth value of each point in the binocular image. This can reduce the amount of calculation for determining the image depth, save computing resources, and improve the efficiency of image depth determination.

[0091] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein program instructions are stored in the memory, and when the processor reads and runs the program instructions, the steps in any of the above implementation methods are executed.

[0092] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above implementation methods are executed.

[0093] The computer-readable storage medium may be a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or any other medium that can store program codes. The storage medium is used to store the program, and the processor executes the program after receiving the execution instruction. The method executed by the electronic terminal of the process definition disclosed in any embodiment of the present invention can be applied to the processor or implemented by the processor.

[0094] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0095] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0097] Alternatively, the present invention may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the present invention may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part.

[0098] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0099] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0100] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining image depth, characterized in that: include: Acquire multiple frames of binocular images, and obtain a depth estimation value of a target frame image based on a preset depth estimation algorithm; wherein the target frame image is an image of any frame in the multiple frames of binocular images; Determining a disparity range of each pixel in the target frame image based on the depth estimation value; and Performing iterative calculations within the disparity range according to a binocular matching algorithm to determine the disparity of each pixel point to complete image depth estimation; Before obtaining the depth estimation value of the target frame image based on the preset depth estimation algorithm, the method further includes: Performing resolution reduction processing on the multiple frames of binocular images to obtain bypass images respectively corresponding to the multiple frames of binocular images; and The bypass image is used as input data of the preset depth estimation algorithm, so as to be used by the preset depth estimation algorithm to obtain the depth estimation value.

2. The method according to claim 1, characterized in that: in, The preset depth estimation algorithm includes: optical flow algorithm, adjacent frame difference method or three-frame difference method; The depth estimation value of the target frame image obtained based on a preset depth estimation algorithm includes: calculating the motion state between adjacent frames in multiple bypass images based on an optical flow algorithm, an adjacent frame difference method or a three-frame difference method to obtain the depth estimation value of the target frame image.

3. The method according to claim 1, characterized in that Before acquiring multiple frames of binocular images and obtaining a depth estimation value of a target frame image based on a preset depth estimation algorithm, the method includes: The multiple frames of binocular images are encoded based on statistical transformation, the pixel grayscale value of the image in the neighborhood window is compared with the pixel grayscale value of the window center, a comparison result is obtained, and the comparison result is mapped into a bit string, and the value of the bit string is used as the statistical transformation value of the center pixel.

4. The method according to claim 1, characterized in that Determining the disparity range of each pixel in the target frame image based on the depth estimation value includes: Calculating a matching cost based on the disparity range; and Cost aggregation is performed on the matching costs, and matching points corresponding to each position are selected from the binocular image to determine the disparity.

5. The method according to claim 4, characterized in that The calculating the matching cost based on the disparity range includes: Based on the encoded results of the multiple frames of binocular images, the Hamming distance between two points in the left and right views of the binocular images is calculated to determine the matching cost.

6. An image depth determination device, characterized in that: include: An estimation module is used to acquire multiple frames of binocular images and obtain a depth estimation value of a target frame image based on a preset depth estimation algorithm; wherein the target frame image is an image of any frame in the multiple frames of binocular images; before obtaining the depth estimation value of the target frame image based on the preset depth estimation algorithm, it also includes: reducing the resolution of the multiple frames of binocular images to obtain bypass images corresponding to the multiple frames of binocular images respectively; and using the bypass images as input data of the preset depth estimation algorithm, so as to be used by the preset depth estimation algorithm to obtain the depth estimation value; a disparity range calculation module, configured to determine a disparity range of each pixel in the target frame image based on the depth estimation value; and The disparity determination module is used to perform iterative calculations within the disparity range according to a binocular matching algorithm to determine the disparity of each pixel point to complete image depth estimation.

7. The device according to claim 6, characterized in that in, The preset depth estimation algorithm includes: optical flow algorithm, adjacent frame difference method or three-frame difference method; The estimation module is specifically used to calculate the motion state between adjacent frames in multiple bypass images based on an optical flow algorithm, an adjacent frame difference method or a three-frame difference method to obtain a depth estimation value of the target frame image.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores program instructions, and when the processor runs the program instructions, the steps in the method according to any one of claims 1 to 5 are executed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the steps in the method according to any one of claims 1 to 5 are executed.

Images