A Parallax Optimization Method, Device and Terminal with High Speed and Low Resource Consumption

Abstract

The present invention discloses a parallax optimization method, device and terminal with high speed and low resource consumption. The method includes: obtaining an initial parallax map, where the initial parallax map is obtained by using a preset parallax algorithm based on a first image and a second image; selecting a color image corresponding to a reference map in the preset parallax algorithm as a target image from the first image and the second image, compressing the target image to obtain a guidance map, where the pixel value of each pixel point in the guidance map is binary; optimizing the initial parallax map according to the guidance map to obtain a target parallax map. The present invention can achieve high-speed and low-consumption parallax optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a parallax optimization method, device and terminal with high speed and low resource consumption. Background Art

[0002] Binocular vision is a technology that obtains parallax by imitating human binoculars according to bionics and then converts it into depth distance information. It is widely used in fields such as driverless, three-dimensional face recognition, simultaneous localization and mapping, etc. The current mainstream binocular vision algorithm is the semi-global stereo matching algorithm (SGM). Most SGMs for hardware platforms use many complex post-processing algorithms for parallax optimization. Although it can improve the accuracy of algorithm processing to a certain extent, it will lead to extremely high computational complexity and increased resource consumption, affecting the calculation speed.

[0003] Therefore, the existing technology still needs to be improved. Summary of the Invention

[0004] In view of the above-mentioned defects of the existing technology, the present invention provides a parallax optimization method, device and terminal with high speed and low resource consumption, aiming to solve the problem of high resource consumption caused by using complex post-processing algorithms for parallax optimization in the existing technology.

[0005] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0006] In the first aspect of the present invention, a parallax optimization method with high speed and low resource consumption is provided. The method includes:

[0007] Obtain an initial parallax map, where the initial parallax map is obtained by using a preset parallax algorithm based on a first image and a second image;

[0008] Select a color image corresponding to the reference map in the preset parallax algorithm as the target image from the first image and the second image, and compress the target image to obtain a guidance map, where the pixel value of each pixel point in the guidance map is binary;

[0009] Optimize the initial parallax map according to the guidance map to obtain a target parallax map.

[0010] In the parallax optimization method with high speed and low resource consumption, the step of compressing the target image to obtain a guidance map includes:

[0011] Divide each channel of the target image into multiple first regions, and the size of each first region is N*N, where N is a positive odd number;

[0012] Binarize the pixel values in each first region according to the pixel value range in each first region to obtain the guidance map.

[0013] The described high-speed and low-resource-consumption parallax optimization method, wherein the step of binarizing the pixel values in each of the first regions according to the pixel value ranges in each of the first regions to obtain the guidance map includes:

[0014] Binarize the pixel values in each of the first regions according to the average value of the maximum pixel value and the minimum pixel value in each of the first regions to obtain the guidance map.

[0015] The described high-speed and low-resource-consumption parallax optimization method, wherein the step of optimizing the initial parallax map according to the guidance map to obtain the target parallax map includes:

[0016] Divide the initial parallax map into a plurality of second regions, the size of each second region being M*M, where M is a positive integer and M = 2*N - 1;

[0017] Output a binary array corresponding to each parallax value in the second region according to each parallax value in the second region and other parallax values in the second region;

[0018] Obtain the weight corresponding to each binary array according to the guidance map and the initial parallax map;

[0019] Perform weighted median filtering on the parallax values in the second region according to the weights corresponding to each binary array corresponding to the second region, update the parallax value at the center of the second region, and obtain the target parallax map.

[0020] The described high-speed and low-resource-consumption parallax optimization method, wherein the step of outputting a binary array corresponding to each parallax value in the second region according to each parallax value in the second region and other parallax values in the second region includes:

[0021] Compare each parallax value in the second region with each parallax value in the second region. If they are equal, output a first value; if they are not equal, output a second value, thereby obtaining the binary array with a size of M*M corresponding to each parallax value in the second region.

[0022] The described high-speed and low-resource-consumption parallax optimization method, wherein the step of obtaining the weight corresponding to each binary array according to the guidance map and the initial parallax map includes:

[0023] Perform guided filtering on the initial parallax map according to the guidance map using a first preset formula to obtain the weight corresponding to each binary array in each second region;

[0024] The first preset formula is:

[0025]

[0026] where ε is a regularization factor, U is the identity matrix, p m,k,i represents the i-th data in the k-th region of size N*N in the m-th binary array of the second region, I k,i represents the vector composed of the pixel values of the i-th pixel point in the k-th region of size N*N in the region corresponding to the second region in the guidance map in three channels, is the mean value of all pixel values in the k-th region of size N*N in the region corresponding to the second region in the guidance map, is the mean value of all data in the k-th region of size N*N in the m-th binary array of the second region, q m is the weight of the binary array corresponding to the m-th disparity value in the second region, I c represents the vector composed of the pixel values of the central pixel point in the region of size M*M corresponding to the second region in the guidance map in three channels.

[0027] The high-speed and low-resource-consumption disparity optimization method, wherein obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map includes:

[0028] Performing guided filtering on the initial disparity map according to the guidance map by using a second preset formula to obtain the weight corresponding to each binary array in each second region;

[0029] The second preset formula is:

[0030]

[0031] where ε is a regularization factor, p m,k,i represents the i-th data in the k-th region of size N*N in the m-th binary array of the second region, I k,i represents the vector composed of the pixel values of the i-th pixel point in the k-th region of size N*N in the region corresponding to the second region in the guidance map in three channels, is the sum of all pixel values in the k-th region of size N*N in the region corresponding to the second region in the guidance map, is the sum of all data in the k-th region of size N*N in the m-th binary array of the second region, q m is the weight of the binary array corresponding to the m-th disparity value in the second region, I cDenote the vector composed of the pixel values of the central pixel point in the region of size M*M corresponding to the second region in the guidance map in three channels. and respectively denote and sum.

[0032] In a second aspect of the present invention, there is provided a parallax optimization device with high speed and low resource consumption, including:

[0033] An initial parallax acquisition module, which is used to acquire an initial parallax map, where the initial parallax map is obtained by using a preset parallax algorithm based on a first image and a second image;

[0034] A color image compression module, which is used to select a color image corresponding to the reference map in the preset parallax algorithm as a target image from the first image and the second image, compress the target image to obtain a guidance map, and the pixel value of each pixel point in the guidance map is binary;

[0035] A target optimization module, which is used to optimize the initial parallax map according to the guidance map to obtain a target parallax map.

[0036] In a third aspect of the present invention, there is provided a terminal, which includes a processor and a computer-readable storage medium communicatively connected to the processor. The computer-readable storage medium is adapted to store a plurality of instructions, and the processor is adapted to call the instructions in the computer-readable storage medium to execute the steps of implementing the high-speed and low-resource-consumption parallax optimization method described in any one of the above.

[0037] In a fourth aspect of the present invention, there is provided a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the high-speed and low-resource-consumption parallax optimization method described in any one of the above.

[0038] Compared with the prior art, the present invention provides a high-speed and low-resource-consumption parallax optimization method, device and terminal. In the high-speed and low-resource-consumption parallax optimization method provided by the present invention, when calculating the parallax between a first image and a second image, a color image corresponding to the reference map in the preset parallax algorithm is selected, the data of the RGB channels of the color image are compressed and cached. After obtaining the initial parallax map according to the first image and the second image, the compressed color image is used as a guidance map to optimize the initial parallax map, so as to achieve parallax optimization. And the data of the color image are compressed and cached, which does not need to occupy a large amount of logical storage resources, and can achieve high-speed and low-consumption parallax optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flowchart of an embodiment of the high - speed and low - resource - consumption parallax optimization method provided by the present invention;

[0040] Figure 2 It is a schematic diagram of the overall framework of an embodiment of the high - speed and low - resource - consumption parallax optimization method provided by the present invention;

[0041] Figure 3 It is a schematic diagram of the process of compressing a target image in an embodiment of the high - speed and low - resource - consumption parallax optimization method provided by the present invention;

[0042] Figure 4 It is a schematic diagram of the process of optimizing an initial parallax map according to a guidance map to obtain a target parallax map in an embodiment of the high - speed and low - resource - consumption parallax optimization method provided by the present invention;

[0043] Figure 5 It is a schematic diagram of the process of obtaining a binary array in an embodiment of the high - speed and low - resource - consumption parallax optimization method provided by the present invention;

[0044] Figure 6 It is a schematic diagram of the structure principle of an embodiment of the high - speed and low - resource - consumption parallax optimization device provided by the present invention;

[0045] Figure 7 It is a schematic diagram of the principle of an embodiment of the terminal provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the objectives, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] The high - speed and low - resource - consumption parallax optimization method provided by the present invention can be applied to a terminal with computing capabilities. The terminal can execute the high - speed and low - resource - consumption parallax optimization method provided by the present invention to calculate the parallax between a first image and a second image. The terminal can be, but is not limited to, various computers, mobile terminals, smart home appliances, wearable devices, etc.

[0048] Embodiment 1

[0049] As Figure 1 shown, in an embodiment of the high - speed and low - resource - consumption parallax optimization method, it includes the steps:

[0050] S100. Obtain an initial parallax map, where the initial parallax map is obtained by using a preset parallax algorithm according to a first image and a second image.

[0051] AsFigure 2 As shown, in order to improve the speed of parallax calculation, the first image and the second image are grayscale images. According to the first image and the second image, the parallax between the first image and the second image can be calculated by using a preset parallax algorithm to generate the initial parallax map. The size of the initial parallax map is the same as that of the first image and the second image. The pixel value of each pixel point in the initial parallax map is the parallax value between the corresponding pixel points in the first image and the second image.

[0052] S200. Select a color image corresponding to the reference image in the preset parallax algorithm from the first image and the second image as the target image, and compress the target image to obtain a guidance map. The pixel value of each pixel point in the guidance map is binary.

[0053] In this embodiment, the color image corresponding to the reference image in the preset parallax algorithm is selected as the target image. Specifically, in the parallax algorithm, an image will be selected as the reference image for calculating the parallax. The reference image can be any one of the first image and the second image. The color image is an RGB image, that is, it has three channels of R, G, and B. Each pixel point has a pixel value on each channel, which are the R value, the G value, and the B value respectively. As Figure 2 As shown, in order to reduce the logical operation and storage consumption, in this embodiment, after compressing the target image to obtain the guidance map, it is cached. Specifically, the process of compressing the target image to obtain the guidance map includes:

[0054] S210. Divide each channel of the target image into multiple first regions, and the size of each first region is N*N, where N is a positive odd number.

[0055] S220. Binarize the pixel values in each first region according to the pixel value range in each first region to obtain the guidance map.

[0056] As described above, the target images all have three channels, and each pixel has a pixel value on each channel. Each channel of the target image can be regarded as a picture. For each channel of the target image, it is divided into multiple adjacent first regions of size N*N. That is to say, each of the first regions includes N*N pixels. The value of N can be set according to the actual situation (such as hardware resources, calculation accuracy, etc. The smaller N is, the greater the occupied logical computing resources, and the larger N is, the lower the calculation accuracy). For example, N can be set to 3, 5, etc. Since in the method provided in this embodiment, the target image is ultimately only used to guide the provision of edge information, the qualities are almost the same in areas where the image does not change much, and the variance and covariance in the area change little or are the same, while there will be a large change at the edge. Therefore, fine RGB pixel values are not required. By binarizing the pixel values in each of the first regions according to the pixel value range in each of the first regions, the guidance map is obtained for optimizing the initial disparity map.

[0057] Binarizing the pixel values in each of the first regions according to the pixel value range in each of the first regions to obtain the guidance map includes:

[0058] Binarizing the pixel values in each of the first regions according to the average value of the maximum pixel value and the minimum pixel value in each of the first regions to obtain the guidance map.

[0059] Specifically, for each of the first regions, the maximum value and the minimum value of the pixel values in the first region can be obtained, and the pixel values in the first region are binarized according to the average value of the two. In this embodiment, as Figure 3 shown, the compression threshold is obtained by adding the maximum pixel value and the minimum pixel value in the first region and then dividing by 2 (shifting one bit to the right), and then the N*N pixel values in the region are compared with the compression threshold. Those greater than or equal to it output 1, and vice versa output 0. In this way, 16-bit or 24-bit RGB data can be compressed into 3 bits for caching, greatly reducing the storage consumption.

[0060] First, cache the pixel values of the three channels of the target image, and then process them to obtain the guidance map. In terms of hardware implementation, for the caching of the target image, two rows of line buffer units are used for caching each of the RGB three channels. Each line buffer is composed of a FIFO (First In First Out module) with the same depth, and its depth depends on the number of columns of the input image, so as to realize outputting the left and right images from left to right to cache two rows of pixel values of the image. For generating the guidance map, find the maximum and minimum values of the pixel values within the first region of size N*N in each channel through sorting, find the boundary value (i.e., the compression threshold in the previous text) of the pixel range within this first region by means of shifting, and then compare each pixel value within the first region with the boundary value. Output 1 if it is greater than or equal to, and output 0 if it is less than. Then perform the same operation on the next first region.

[0061] It is not difficult to see that through the operation in step S200, the pixel values of each pixel point of the target image in each channel will be processed into binary values of 0 and 1, thus generating a new image in which the pixel values of each pixel point are either 0 or 1, denoted as the guidance map.

[0062] Please refer to again Figure 1 , the high-speed and low-resource-consumption disparity optimization method provided in this embodiment, after generating the guidance map, further includes the steps:

[0063] S300. Optimize the initial disparity map according to the guidance map to obtain the target disparity map.

[0064] As Figure 4 shown, after obtaining the guidance map and the initial disparity map, first process the initial disparity map to obtain a plurality of binary arrays, then calculate the weight of each array according to the guidance map, and finally perform weighted median filtering calculation according to the weights of each binary array and the initial disparity map to obtain the target disparity map. Specifically, the step of optimizing the initial disparity map according to the guidance map to obtain the target disparity map includes the steps:

[0065] S310. Divide the initial disparity map into a plurality of second regions, each of the second regions having a size of M*M, where M is a positive integer and M = 2*N - 1.

[0066] S320. Output the binary array corresponding to each disparity value within the second region according to each disparity value within the second region and other disparity values within the second region.

[0067] The pixel value of each pixel point in the initial disparity map is a disparity value. The initial disparity map is divided into a plurality of adjacent second regions, and the size of each second region is M×M. That is to say, there are M×M disparity values in each second region, M = 2×N - 1, and the value of M can be selected as 5, 9, etc.

[0068] A binary array is generated corresponding to each disparity value in each second region. Specifically, the step of outputting the binary array corresponding to each disparity value in the second region according to each disparity value in the second region and other disparity values in the second region includes:

[0069] Compare each disparity value in the second region with each disparity value in the second region. If they are equal, output a first value; if they are not equal, output a second value, so as to obtain the binary arrays with the size of M×M respectively corresponding to each disparity value in the second region.

[0070] For each disparity value in the second region, compare it with each disparity value in the second region. If they are equal, output a first value, such as 1; otherwise, output a second value, such as 0. In this way, for each disparity value in the second region, a binary array composed of 1 and 0 will be generated. That is to say, for each second region, M×M binary arrays will be generated.

[0071] As Figure 5 shown, specifically, in terms of hardware implementation, generating the binary array can be realized by a selection array module. Taking N = 3 and M = 5 as an example, use pab to represent the disparity value at the a-th row and b-th column in a second region. There is an enable signal sel in each selection array. First, set the sels of 25 selection arrays for p00 - p44 respectively, and then compare them with the 25 disparity values of p00 - p44 at the same time. If they are equal, control the corresponding selector (MUX) to output 1; otherwise, output 0.

[0072] S330. Obtain the weight corresponding to each binary array according to the guidance map and the initial disparity map.

[0073] In the traditional guidance filtering formula, there are a large number of matrix operations and division operations, including a complex inverse process. However, in the method provided in this embodiment, since the image data of the RGB three channels is compressed and the data of each channel is 0 and 1, the influence of the covariance matrix can be ignored. Therefore, in a possible implementation manner of this embodiment, the step of obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map includes:

[0074] Performing guided filtering on the initial disparity map according to the guiding map using a first preset formula to obtain the weights corresponding to each binary array in each of the second regions;

[0075] The first preset formula is:

[0076]

[0077] where ε is a regularization factor, U is an identity matrix, p m,k,i represents the i-th data in the k-th region of size N×N in the m-th binary array of the second region, and I k,i represents the vector composed of the pixel values of the i-th pixel point in the k-th region of size N×N in the region corresponding to the second region in the guiding map in three channels, is the mean value of all pixel values in the k-th region of size N×N in the region corresponding to the second region in the guiding map, is the mean value of all data in the k-th region of size N×N in the m-th binary array of the second region, q m is the weight of the binary array corresponding to the m-th disparity value in the second region, and I c represents the vector composed of the pixel values of the central pixel point in the region of size M×M corresponding to the second region in the guiding map in three channels.

[0078] From the explanations of each term in the first preset formula in the foregoing, it is not difficult to see that for each of the second regions, there are M×M binary arrays, each binary array has a size of M×M, then each binary array can be divided into N×N regions of size N×N, and each region includes N×N data. Therefore, for any m with a value range from 1 to M×M, the value range of k is from 1 to N×N, that is to say, each binary array can calculate N×N a m, k and N×N b m,k , after taking the mean of all a m,k and multiplying it by the central pixel value in the region corresponding to the second region in the guiding map, and then adding the mean of all b m,k , the weight of the binary array corresponding to the m-th disparity value in the second region can be obtained. After verification, the calculation using the first preset formula has almost the same calculation accuracy as the traditional guided filtering algorithm. Therefore, the method provided in this embodiment has successfully reduced the operation significantly on the algorithm level while ensuring the accuracy.

[0079] Further, in this embodiment, the weights of each of the binary arrays are ultimately for subsequent weighted median filtering. If the first preset formula is used to calculate the weights, a large number of fixed-point dividers are required at the hardware level, which will seriously affect the accuracy. Therefore, in another possible implementation, the operation of division is avoided by expanding the same multiple, that is, the weights corresponding to each of the binary arrays are obtained according to the guidance map and the initial disparity map, including:

[0080] Guided filtering is performed on the initial disparity map according to the guidance map using a second preset formula to obtain the weights corresponding to each of the binary arrays in each of the second regions;

[0081] The second preset formula is:

[0082]

[0083] where ε is a regularization factor, p m, k,i represents the i-th data in the k-th N*N region of the m-th binary array in the second region, and I k,i represents the vector composed of the pixel values of the i-th pixel point in the k-th N*N region in the region corresponding to the second region in the guidance map in three channels, is the sum of all pixel values in the k-th N*N region in the region corresponding to the second region in the guidance map, is the sum of all data in the k-th N*N region of the m-th binary array in the second region, and q m is the weight of the binary array corresponding to the m-th disparity value in the second region, and I c represents the vector composed of the pixel values of the central pixel point in the M*M region corresponding to the second region in the guidance map in three channels, and respectively represent the sum of and for summation.

[0084] S340. Perform weighted median filtering on the second region according to the weights corresponding to each of the binary arrays corresponding to the second region, update the disparity value at the center of the second region, and obtain the target disparity map.

[0085] Specifically, according to the weights corresponding to each of the binary arrays corresponding to the second region, a weighted median filter is performed on the disparity values within the second region to update the disparity value at the center of the second region. First, for the same disparity values within the target region, only one of their weights is retained. Subsequently, the N*N weights within the target region are sorted according to the weight magnitudes. Finally, the disparity value corresponding to the median weight is used as the target disparity value, and the disparity value at the center point of the target region is updated to the target disparity value. That is to say, for each of the second regions, the disparity value at the center point of the region of size N*N within the second region will be updated.

[0086] In terms of hardware implementation, for the disparity and RGB compressed data, each is subjected to regional caching operations by four row buffer modules, and these operations are synchronous. The calculation after caching consists of two-level pipelines. The first-level calculation uses the N*N pixel points within the disparity region and the weights with the guidance map as inputs. The second level first finds the same disparity values within the target region, retains only one of the same disparity values, and assigns the rest to 0. Then, the weights are sorted to find the disparity value corresponding to the median as the output.

[0087] In summary, the present embodiment provides a disparity optimization method with high speed and low resource consumption. When calculating the disparity between the first image and the second image, a color image corresponding to the reference image in the preset disparity algorithm is selected, the data of the RGB channels of the color image is compressed and cached. After the initial disparity map is calculated based on the first image and the second image, the compressed color image is used as the guidance map to optimize the initial disparity map, which can achieve disparity optimization. Moreover, the data of the color image is cached after compression, without occupying a large amount of logical storage resources, and can achieve high-speed and low-consumption disparity optimization.

[0088] It should be understood that although the steps in the flowcharts given in the accompanying drawings of the present invention are shown in sequence according to the indications of the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0089] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or an external cache. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0090] Embodiment 2

[0091] Based on the above embodiments, the present invention also correspondingly provides a parallax optimization device with high speed and low resource consumption, as Figure 6 shown. The parallax optimization device with high speed and low resource consumption includes:

[0092] An initial parallax acquisition module, which is used to acquire an initial parallax map. Among them, the initial parallax map is obtained by using a preset parallax algorithm based on a first image and a second image, specifically as described in Embodiment 1;

[0093] A color image compression module, which is used to select a color image corresponding to one of the first image and the second image as a target image, compress the target image, and obtain a guidance map. The pixel value of each pixel point in the guidance map is binary, specifically as described in Embodiment 1;

[0094] A target optimization module, which is used to optimize the initial parallax map according to the guidance map to obtain a target parallax map, specifically as described in Embodiment 1.

[0095] Embodiment 3

[0096] Based on the above embodiments, the present invention also correspondingly provides a terminal, asFigure 7 As shown, the terminal includes a processor 10 and a memory 20. Figure 7 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0097] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as the hard disk or memory of the terminal. In some other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 20 may also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, a parallax optimization program 30 with high speed and low resource consumption is stored on the memory 20, and the parallax optimization program 30 with high speed and low resource consumption can be executed by the processor 10, so as to implement the parallax optimization method with high speed and low resource consumption in the present application.

[0098] In some embodiments, the processor 10 may be a Central Processing Unit (CPU), a microprocessor or other chips, and is used to run the program code stored in the memory 20 or process data, such as executing the parallax optimization method with high speed and low resource consumption, etc.

[0099] In one embodiment, when the processor 10 executes the parallax optimization program 30 in the memory 20, the following steps are implemented:

[0100] Obtain an initial parallax map, where the initial parallax map is obtained by using a preset parallax algorithm according to a first image and a second image;

[0101] Select a color image of a reference map in the preset parallax algorithm as a target image from the first image and the second image, and compress the target image to obtain a guidance map, where the pixel value of each pixel point in the guidance map is binary;

[0102] Optimize the initial parallax map according to the guidance map to obtain a target parallax map.

[0103] Wherein, the compressing the target image to obtain a guidance map includes:

[0104] Divide each channel of the target image into a plurality of first regions, where the size of each first region is N*N, and N is a positive odd number;

[0105] Binarize the pixel values in each first region according to the pixel value range in each first region to obtain the guidance map.

[0106] Among them, the step of binarizing the pixel values in each first region according to the pixel value range in each first region to obtain the guidance map includes:

[0107] Binarize the pixel values in each first region according to the average value of the maximum pixel value and the minimum pixel value in each first region to obtain the guidance map.

[0108] Among them, the step of optimizing the initial disparity map according to the guidance map to obtain the target disparity map includes:

[0109] Divide the initial disparity map into a plurality of second regions, where the size of each second region is M*M, M is a positive integer, and M = 2*N - 1;

[0110] Output a binary array corresponding to each disparity value in the second region according to each disparity value in the second region and other disparity values in the second region;

[0111] Obtain the weight corresponding to each binary array according to the guidance map and the initial disparity map;

[0112] Perform weighted median filtering on the disparity values in the second region according to the weight corresponding to each binary array corresponding to the second region, update the disparity value at the center of the second region, and obtain the target disparity map.

[0113] Among them, the step of outputting a binary array corresponding to each disparity value in the second region according to each disparity value in the second region and other disparity values in the second region includes:

[0114] Compare each disparity value in the second region with each disparity value in the second region. If they are equal, output a first value; if they are not equal, output a second value, and obtain the binary array with a size of M*M corresponding to each disparity value in the second region.

[0115] Among them, the step of obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map includes:

[0116] Perform guided filtering on the initial disparity map according to the guidance map using a first preset formula to obtain the weight corresponding to each binary array in each second region;

[0117] The first preset formula is as follows:

[0118]

[0119] where ε is the regularization factor, U is the identity matrix, p m,k,i represents the i-th data in the k-th region of size N×N in the m-th binary array of the second region, and I k,i represents the vector composed of the pixel values of the i-th pixel point in the k-th region of size N×N in the region corresponding to the second region in the guidance map in three channels, is the mean value of all pixel values in the k-th region of size N×N in the region corresponding to the second region in the guidance map, is the mean value of all data in the k-th region of size N×N in the m-th binary array of the second region, and q m is the weight of the binary array corresponding to the m-th disparity value in the second region, and I c represents the vector composed of the pixel values of the central pixel point in the region of size M×M corresponding to the second region in the guidance map in three channels.

[0120] Among them, obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map includes:

[0121] Performing guided filtering on the initial disparity map according to the guidance map using a second preset formula to obtain the weight corresponding to each binary array in each second region;

[0122] The second preset formula is as follows:

[0123]

[0124] where ε is the regularization factor, p m,k,i represents the i-th data in the k-th region of size N×N in the m-th binary array of the second region, and I k,i represents the vector composed of the pixel values of the i-th pixel point in the k-th region of size N×N in the region corresponding to the second region in the guidance map in three channels, is the sum of all pixel values in the k-th region of size N×N in the region corresponding to the second region in the guidance map, is the sum of all data in the k-th region of size N×N in the m-th binary array of the second region, and q m is the weight of the binary array corresponding to the m-th disparity value in the second region, and I cDenote the vector composed of the pixel values of the central pixel point in the region of size M*M corresponding to the second region in the guidance map in three channels. and respectively denote and the summation.

[0125] Embodiment 4

[0126] The present invention also provides a computer-readable storage medium, in which one or more programs are stored, and the one or more programs can be executed by one or more processors to implement the steps of the high-speed and low-resource-consumption parallax optimization method as described above.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A parallax optimization method with high speed and low resource consumption, characterized in that, The method includes: Obtaining an initial disparity map, where the initial disparity map is obtained by using a preset disparity algorithm based on a first image and a second image; Selecting a color image corresponding to a reference map in the preset disparity algorithm from the first image and the second image as a target image, and compressing the target image to obtain a guidance map, where the pixel value of each pixel point in the guidance map is binary; Optimizing the initial disparity map according to the guidance map to obtain a target disparity map; The compressing the target image to obtain a guidance map includes: Dividing each channel of the target image into a plurality of first regions, where the size of each first region is N*N, and N is a positive odd number; Binarizing the pixel values in each first region according to the pixel value range in each first region to obtain the guidance map; The optimizing the initial disparity map according to the guidance map to obtain a target disparity map includes: Dividing the initial disparity map into a plurality of second regions, where the size of each second region is M*M, M is a positive odd number, and M = 2*N - 1; Outputting a binary array corresponding to each disparity value in the second region according to each disparity value in the second region and other disparity values in the second region; Obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map; Performing weighted median filtering on the disparity values in the second region according to the weight corresponding to each binary array corresponding to the second region, updating the disparity value at the center of the second region, and obtaining the target disparity map.

2. The parallax optimization method with high speed and low resource consumption according to claim 1, characterized in that, The binarizing the pixel values in each first region according to the pixel value range in each first region to obtain the guidance map includes: Binarizing the pixel values in each first region according to the average value of the maximum pixel value and the minimum pixel value in each first region to obtain the guidance map.

3. The parallax optimization method with high speed and low resource consumption according to claim 1, characterized in that, The outputting a binary array corresponding to each disparity value in the second region according to each disparity value in the second region and other disparity values in the second region includes: Comparing each disparity value in the second region with each disparity value in the second region. If they are equal, output a first value. If they are not equal, output a second value, and obtaining the binary array with a size of M*M corresponding to each disparity value in the second region.

4. The parallax optimization method with high speed and low resource consumption according to claim 1, characterized in that, The obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map includes: Performing guided filtering on the initial disparity map according to the guidance map by using a first preset formula to obtain the weight corresponding to each binary array in each second region; The first preset formula is: ； ； ； where ε is a regularization factor and U is the identity matrix, denotes the i-th data in the k-th region of size N×N in the m-th binary array of the second region, Ik , i denotes the vector formed by the pixel values of the i-th pixel point in the k-th region of size N×N in the region corresponding to the second region in the guidance map in three channels, is the mean value of all pixel values in the k-th region of size N×N in the region corresponding to the second region in the guidance map, is the mean value of all data in the k-th region of size N×N in the m-th binary array of the second region, qm is the weight of the binary array corresponding to the m-th disparity value in the second region, Ic denotes the vector formed by the pixel values of the central pixel point in the region of size M×M corresponding to the second region in the guidance map in three channels.

5. The parallax optimization method with high speed and low resource consumption according to claim 1, characterized in that, The obtaining the weight corresponding to each binary array according to the guidance map and the initial disparity map includes: Performing guided filtering on the initial disparity map according to the guidance map by using a second preset formula to obtain the weight corresponding to each binary array in each second region; The second preset formula is: ； ； ； where ε is a regularization factor, represents the i-th data in the k-th region of size N*N in the m-th binary array of the second region, represents the vector composed of the pixel values of the i-th pixel point in the k-th region of size N*N in the region corresponding to the second region in the guidance map in three channels, is the sum of all pixel values in the k-th region of size N*N in the region corresponding to the second region in the guidance map, is the sum of all data in the k-th region of size N*N in the m-th binary array of the second region, is the weight of the m-th binary array corresponding to the m-th disparity value in the second region, and Ic represents the vector composed of the pixel values of the central pixel point in three channels in the region of size M*M corresponding to the second region in the guidance map, and respectively represent the sum of and by summation.

6. A parallax optimization device with high speed and low resource consumption, characterized in that, The high-speed and low-resource-consumption parallax optimization device is applied to the high-speed and low-resource-consumption parallax optimization described in any one of claims 1-5, and includes: An initial parallax acquisition module, which is used to acquire an initial parallax map. Among them, the initial parallax map is obtained by using a preset parallax algorithm based on a first image and a second image; A color image compression module, which is used to select a color image corresponding to the reference map in the preset parallax algorithm from the first image and the second image as a target image, compress the target image, and obtain a guidance map. The pixel value of each pixel point in the guidance map is binary; A target optimization module, which is used to optimize the initial parallax map according to the guidance map to obtain a target parallax map.

7. A terminal, characterized in that, The terminal includes: a processor and a computer-readable storage medium communicatively connected to the processor. The computer-readable storage medium is suitable for storing multiple instructions, and the processor is suitable for calling the instructions in the computer-readable storage medium to execute the steps of implementing the high-speed and low-resource-consumption parallax optimization method described in any one of claims 1-5 above.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the high-speed and low-resource-consumption parallax optimization method described in any one of claims 1-5.

Images