Image processing method, intelligent terminal and computer readable storage medium

Abstract

The present invention discloses an image processing method, an intelligent terminal and a computer-readable storage medium, the method comprising: providing a reference image and an original image for the same target scene, and performing disparity processing on the original image to generate a corresponding first disparity map; generating a second disparity map corresponding to a preset target resolution according to the first disparity map; processing a first depth map corresponding to the first disparity map according to the reference image to generate a second depth map; and determining a third disparity map according to the second depth map and the second disparity map. The present invention can retain more edge details on the basis of generating a high-resolution disparity map, retain the disparity accuracy of the original disparity map to the greatest extent, and has higher computational efficiency.

Description

Technical Field

[0001] The present invention relates to the field of stereoscopic vision technology, and in particular to an image processing method, an intelligent terminal and a computer-readable storage medium. Background Art

[0002] At present, many smart terminal cameras use dual cameras or multiple cameras in the visible light frequency domain to take pictures. Taking dual camera photography as an example, there are differences between the two images taken by the dual cameras. On the one hand, there are differences in the imaging of the dual cameras, such as differences caused by factors such as focal length, lens distortion, white balance, exposure level, noise level, and shooting field of view. On the other hand, physical factors such as occlusion, hollowing, mirror reflection, and optical refraction of transparent objects between the shooting objects will also cause great differences in the imaging of the same object in the two cameras. Therefore, it is necessary to perform pixel-level matching on the images of the same scene taken by the dual cameras to obtain the pixel coordinate offset of each pixel under the two cameras. This process is also called binocular disparity estimation. However, due to the above reasons, binocular disparity estimation is difficult. The calculated original disparity map often shows uneven disparity of the subject target and the presence of many holes and bright spots. Holes are areas with a disparity value of 0, and bright spots are areas with abnormally high disparity.

[0003] In order to improve the accuracy of binocular disparity estimation, the original disparity map needs to be refined. There are two main methods for refining disparity maps in the existing technology. One is to upsample and refine the image based on local or semi-global filters such as joint bilateral filtering, guided filtering, weighted least squares filtering, and global smoothing filtering with a fixed window size. This method is too rough and the effect is generally poor. The other method uses adaptive weighted median filtering, fast bilateral solution filtering, and disparity map super-resolution and image refinement based on deep convolutional networks. Although this method has better results, it is computationally intensive and therefore time-consuming. Summary of the invention

[0004] The main purpose of the present invention is to provide an image processing method, an intelligent terminal and a computer-readable storage medium, aiming to solve the problem of low refinement of disparity maps in the prior art.

[0005] To achieve the above object, the present invention provides an image processing method, which comprises the following steps:

[0006] Providing a reference image and an original image for the same target scene, and performing disparity processing on the original image to generate a corresponding first disparity map;

[0007] Generating a second disparity map corresponding to a preset target resolution according to the first disparity map;

[0008] Processing a first depth map corresponding to the first disparity map according to the reference image to generate a second depth map;

[0009] A third disparity map is determined according to the second depth map and the second disparity map.

[0010] Optionally, the image processing method, wherein providing a reference image and an original image for the same target scene, specifically comprises:

[0011] Controlling a preset binocular camera to shoot the target scene to obtain a binocular image, wherein the binocular image includes a left-eye image and a right-eye image;

[0012] The left eye image or the right eye image is selected as a reference image, and the other binocular image is used as an original image.

[0013] Optionally, the image processing method, wherein the performing disparity processing on the original image to generate the corresponding first disparity map, specifically includes:

[0014] According to the reference image, performing disparity calculation on the original image to generate a corresponding fourth disparity map;

[0015] Holes are filled in the fourth disparity image to generate a first disparity image corresponding to the fourth disparity image.

[0016] Optionally, the image processing method, wherein hole filling is performed on the fourth disparity map to generate a first disparity map corresponding to the fourth disparity map, specifically comprises:

[0017] Filtering hole pixels in the fourth disparity map according to a preset hole pixel value;

[0018] Determine corresponding neighboring pixels according to the coordinates of the hole pixel, wherein the neighboring pixels are pixels that are adjacent to the hole pixel and are not hole pixels;

[0019] Calculating a filling value corresponding to the hole pixel according to the hole pixel and the neighboring pixels;

[0020] According to the filling value, pixel values ​​of the hole pixels are assigned to generate a first disparity map.

[0021] Optionally, the image processing method, before screening the hole pixels in the fourth disparity map according to the preset hole pixel values, further comprises:

[0022] Binarizing the fourth disparity map to generate a binary disparity map;

[0023] determining a bright spot area of ​​the fourth disparity map according to an area and a mean value of a connected domain in the binary disparity map;

[0024] According to the hole pixel value, each pixel in the bright spot area is converted into a hole pixel.

[0025] Optionally, the image processing method, wherein generating a second disparity map corresponding to a preset target resolution according to the first disparity map, specifically includes:

[0026] Generating a third depth map corresponding to the first disparity map according to preset camera parameters;

[0027] Upsampling the third depth map according to a preset target resolution to generate a fourth depth map;

[0028] The fourth depth map is converted into a second disparity map according to preset camera parameters.

[0029] Optionally, the image processing method, wherein generating a third depth map corresponding to the first disparity map according to preset camera parameters, specifically includes:

[0030] Performing enhancement processing on the reference image to generate an intermediate image;

[0031] performing filtering processing on the first disparity map according to the intermediate image to generate a fifth disparity map;

[0032] A third depth map corresponding to the fifth disparity map is generated according to the camera parameters.

[0033] Optionally, in the image processing method, wherein the reference image includes a grayscale image and a color image; the step of performing enhancement processing on the reference image to generate an intermediate image specifically includes:

[0034] When the reference image is a grayscale image, performing noise reduction processing on the reference image to generate the intermediate image;

[0035] When the reference image is a color image, noise reduction processing is performed on the reference image to generate a noise-reduced image, and color enhancement is performed on the noise-reduced image to generate an intermediate image.

[0036] Optionally, the image processing method, wherein the step of processing the first depth map corresponding to the first disparity map according to the reference image to generate the second depth map, specifically includes:

[0037] Upsampling the intermediate image according to the target resolution to generate a target image;

[0038] Calculate the hash value corresponding to each target pixel in the target image, and use the coordinate value of the target pixel as an index value to generate a corresponding target hash value table;

[0039] Calculating a mapping map corresponding to the target image according to the target hash value table;

[0040] A confidence map corresponding to the first depth map is calculated, and a second depth map corresponding to the mapping map is calculated based on the confidence map and the first depth map.

[0041] Optionally, the image processing method, wherein the step of calculating the hash value corresponding to each target pixel in the target image and using the coordinate value of the target pixel as an index value to generate a corresponding target hash value table, specifically includes:

[0042] Dividing the target image according to a preset division interval to generate a plurality of divided images;

[0043] Calculating a hash value corresponding to each target pixel in the target image according to the division coordinates and the division pixel values ​​of the plurality of division images;

[0044] Matching the index value with the hash value, and creating an initial hash table corresponding to the hash value;

[0045] The initial hash table is updated according to the third depth map to generate a target hash table.

[0046] Optionally, the image processing method, wherein the updating of the initial hash table according to the third depth map to generate a target hash table specifically includes:

[0047] Calculating a support mask corresponding to the target image according to the third depth map and the target resolution;

[0048] According to the support mask, each target pixel of the target image is sampled and marked to generate sampled pixels and non-sampled pixels, wherein the non-sampled pixels are pixels that are not sampled and marked among the target pixels;

[0049] The initial hash table is updated according to the coordinate values ​​of the sampled pixels and the corresponding hash values ​​to generate a target hash table.

[0050] Optionally, in the image processing method, the step of calculating a support mask corresponding to the target image according to the third depth map and the target resolution specifically includes:

[0051] Performing edge extraction on the third depth map to generate a first edge map;

[0052] Upsampling the first edge map according to a preset target resolution to generate a second edge map;

[0053] According to a preset truncation threshold, threshold truncation is performed on each pixel value in the second edge image to generate a binary mask;

[0054] The binary mask is morphologically expanded to generate a support mask corresponding to the target image.

[0055] Optionally, the image processing method, wherein the step of performing sampling marking on each target pixel of the target image according to the support mask to generate sampled pixels and non-sampled pixels, specifically comprises:

[0056] According to a preset high-density sampling value, a target pixel in the target image corresponding to the support mask is sampled and marked, and according to a preset low-density sampling value, a target pixel in the target image corresponding to the support mask is sampled and marked to generate a sampling pixel;

[0057] Pixels in the target pixels that are not marked as sampling pixels are regarded as non-sampling pixels.

[0058] Optionally, the image processing method, wherein the updating of the initial hash table according to the coordinate values ​​of the sampled pixels and the corresponding hash values ​​to generate the target hash table specifically includes:

[0059] Calculating a sampling hash value corresponding to the sampled pixel;

[0060] Traversing the sampled pixels, and determining whether there exists in the initial hash table a hash value identical to the sampled hash value corresponding to the sampled pixel;

[0061] If not, the coordinate value of the sampled pixel and the corresponding sampled hash value are added to the initial hash table to generate the target hash table.

[0062] Optionally, in the image processing method, the step of calculating a mapping image corresponding to the target image according to the target hash value table specifically includes:

[0063] When the target pixel is a sampling pixel, according to the target hash table, the index value corresponding to the sampling pixel is used as the mapping value corresponding to the sampling pixel;

[0064] When the target pixel is a non-sampled pixel, determining a mapping value corresponding to the non-sampled pixel according to whether a hash value corresponding to the non-sampled pixel exists in the target hash table;

[0065] A mapping image corresponding to the target image is generated according to the coordinates of the sampled pixels, the coordinates of the non-sampled pixels and the mapping values.

[0066] Optionally, the image processing method, wherein determining the mapping value corresponding to the non-sampled pixel according to whether there is a hash value corresponding to the non-sampled hash value corresponding to the non-sampled pixel in the target hash table, specifically includes:

[0067] For each non-sampled pixel, when the target hash table has a hash value corresponding to the non-sampled pixel, calculate an index value corresponding to the hash value, and use the index value as a mapping value corresponding to the non-sampled pixel;

[0068] When the target hash table does not contain the hash value corresponding to the non-sampled pixel, a preset null value is used as the mapping value corresponding to the non-sampled pixel.

[0069] Optionally, the image processing method, wherein the calculating a confidence map corresponding to the first depth map, and calculating a second depth map corresponding to the mapping map according to the confidence map and the first depth map, specifically includes:

[0070] Inverting the second edge map to generate a confidence map corresponding to the first depth map, and performing a bilateral solution on the mapping map according to the confidence map and the first depth map to generate depth values ​​corresponding to each index value in the target hash table;

[0071] Generate a fifth depth map corresponding to the target pixel according to the mapping value in the mapping map and the depth value corresponding to the index value, wherein when the mapping value is a null value, the pixel value corresponding to the target pixel in the fifth depth map is a null value;

[0072] According to the target image, depth pixels whose pixel values ​​in the fifth depth map are null values ​​are used as hole pixels, and the holes are filled to generate a sixth depth map;

[0073] The pixel value of the sixth depth map is updated according to the updated pixel value to generate a second depth map, wherein the updated pixel value is a maximum value of pixel values ​​in the first depth map and the sixth depth map when corresponding to the same pixel.

[0074] Optionally, the image processing method, wherein determining a third disparity map according to the second depth map and the second disparity map, specifically includes:

[0075] The second disparity map is processed according to the second depth map to obtain a third disparity map.

[0076] Optionally, the image processing method, wherein the step of processing the second disparity map according to the second depth map to obtain a third disparity map, specifically includes:

[0077] According to the second depth map, smoothing filtering is performed on the second disparity map to obtain a third disparity map.

[0078] Optionally, the image processing method, after determining the third disparity map according to the second depth map and the second disparity map, further comprises:

[0079] Three-dimensional reconstruction is performed according to the third disparity map, the reference image and / or the original image to obtain a three-dimensional scene corresponding to the target scene.

[0080] In addition, to achieve the above-mentioned purpose, the present invention also provides an intelligent terminal, wherein the intelligent terminal includes: a memory, a processor, and an image processing program stored in the memory and executable on the processor, and when the image processing program is executed by the processor, the steps of the image processing method described above are implemented.

[0081] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method as described above are implemented.

[0082] The present invention first pre-processes and up-samples the low-resolution disparity map, and then converts the disparity map into a depth map. The disparity map and the depth map are processed at a higher resolution, thereby improving the fineness of the disparity map. In addition, a stronger depth edge mask is also used, and the fast bilateral solution filter is combined with the depth edge mask for global filtering processing to obtain a depth map after edge refinement. Therefore, the third disparity map finally output by the present invention not only has high resolution, but also retains more edge information and has a high degree of refinement. In addition, the present invention also improves the fast bilateral solution. The current bilateral solution is a global solution algorithm, and its calculation time complexity is very high. The calculation time on the terminal device is too long and the energy consumption is too high to meet the application requirements. In some scenes, it will cause large-area disparity value diffusion, and the disparity map super-resolution and refinement effects are not good. The present invention adopts bilateral hash sparse mapping, which reduces the amount of data calculated while ensuring the accuracy of the hash table, and improves the calculation speed. On this basis, bilateral solution can reduce the requirements for equipment, so it is more universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a simplified flow chart of a preferred embodiment of the image processing method provided by the present invention;

[0084] Figure 2 It is an overall flow chart of a preferred embodiment of the image processing method provided by the present invention;

[0085] Figure 31 is an intermediate image of target resolution (left) and a corresponding fourth disparity map of low resolution (right) in a preferred embodiment of the image processing method of the present invention;

[0086] Figure 4 1 is a first disparity map (left) and a fifth disparity map (right) in a preferred embodiment of the image processing method of the present invention;

[0087] Figure 5 2 is a second edge map (left) and a support mask (right) in a preferred embodiment of the image processing method of the present invention;

[0088] Figure 6 2 are the second depth map (left) and the third disparity map (right) in a preferred embodiment of the image processing method of the present invention;

[0089] Figure 7 Schematic diagram of the operating environment of a preferred embodiment of the intelligent terminal of the present invention. DETAILED DESCRIPTION

[0090] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0091] The image processing method described in the preferred embodiment of the present invention is as follows: Figure 1 and Figure 2 As shown, the image processing method comprises the following steps:

[0092] Step S100 : providing a reference image and an original image for the same target scene, and performing disparity processing on the original image to generate a corresponding first disparity map.

[0093] Specifically, see Figure 3 The execution subject of the image processing method is an image processing program installed in a mobile terminal. The mobile terminal has several cameras. Therefore, when the user controls the mobile terminal to take pictures, several images will be obtained, which are respectively marked as image 1, image 2 and image 3. One of the images is randomly selected as a reference image. In this embodiment, image 1 is used as the reference image. Then, a traditional disparity algorithm is used to calculate the disparity between image 1 and image 2, so as to generate a first disparity map and perform subsequent processing.

[0094] Controlling a preset binocular camera to shoot the target scene to obtain a binocular image, wherein the binocular image includes a left-eye image and a right-eye image;

[0095] The left eye image or the right eye image is selected as a reference image, and the other binocular image is used as an original image.

[0096] Specifically, binocular stereo matching has always been a research hotspot in binocular vision. A binocular camera shoots left and right viewpoint images of the same scene, and uses a stereo matching algorithm to obtain a disparity map, and then obtains a depth map. When the application scenario is that a binocular camera shoots a target scene, a binocular image, i.e., a left eye image and a right eye image, is obtained through the left and right cameras. Then, one of the left eye image or the right eye image is selected as the reference image, and the other binocular image is used as the original image, and then the first disparity map between the binocular images is calculated. The method for calculating the first disparity map can use a stereo matching algorithm, a normalized cross-correlation matching method, and the like.

[0097] Furthermore, step S100 includes:

[0098] Step A10: performing disparity calculation on the original image according to the reference image to generate a corresponding fourth disparity map.

[0099] Specifically, by using a traditional depth sensor or a disparity matching algorithm, etc., the disparity of the image 1 and the image 2 may be calculated, thereby generating a fourth disparity map.

[0100] For example, the common Sum of Absolute Differences (SAD) algorithm scans image 1 sequentially, selects an anchor point, constructs a small window, similar to a convolution kernel; covers image 1 with the window, selects all pixels in the window coverage area; similarly covers image 2 with the window, and selects pixels in the coverage area; subtracts the left coverage area from the right coverage area, and calculates the absolute value of the grayscale difference of all pixels; finds the window with the smallest SAD in the range of image 2 as the best matching pixel block of the anchor point of image 1, thereby generating the fourth disparity map of image 1 and image 2.

[0101] Step A20: Filling holes in the fourth disparity map to generate a first disparity map corresponding to the fourth disparity map.

[0102] Specifically, since the first disparity map is obtained in a conventional manner, there are problems such as bright spots, holes, and low resolution. Therefore, by filling holes on the basis of the first disparity map, a disparity map with better image quality can be obtained, which is beneficial to subsequent refinement work. The hole filling methods currently used include difference filling, etc., which will not be described one by one here.

[0103] Further, step A10 includes:

[0104] Step B10: screening hole pixels in the fourth disparity map according to preset hole pixel values.

[0105] Specifically, in this embodiment, the hole pixel value is 0, and according to the hole pixel value, pixel values ​​of each pixel in the fourth disparity map are compared to determine the hole pixel therein.

[0106] Step B20, determining corresponding neighboring pixels according to the coordinates of the hole pixel, wherein the neighboring pixels are pixels that are adjacent to the hole pixel and are not hole pixels.

[0107] Specifically, according to the coordinates of the hole pixel, the neighboring pixels of each hole pixel are searched in turn. In this embodiment, the search method adopted is to search in four directions: up, down, left, and right. The pixels with non-zero disparity values ​​closest to the coordinates of the hole pixel in these four directions are searched, p1, p2, p3, and p4.

[0108] Step B30: Calculate a filling value corresponding to the hole pixel according to the hole pixel and the neighboring pixels.

[0109] Specifically, according to the resolution of the disparity map, the image 1 or the image 2 is converted into a reference image with the same resolution. Then, according to the reference image, the absolute value of the color difference between the hole pixel and the adjacent pixel is determined, and the reciprocal of the absolute value is used as the weight to weight the filling value of the hole pixel p0. The calculation formula is as follows:

[0110]

[0111] Among them, D(p i ) represents the neighboring pixel p i The original disparity value in the fourth disparity map D; C(p i ) represents the neighboring pixel p i Corresponding to the color value in the reference image C, when the image C is a three-channel RGB image, this item is expressed as the accumulation of the difference of each channel; D'(p0) represents the filling value corresponding to the hole pixel p0.

[0112] Step B40: assigning pixel values ​​of the hole pixels according to the filling value to generate a first disparity map.

[0113] Specifically, see Figure 3 , assigning the filling value to the pixel value of the hole pixel, thereby completing the hole filling of the fourth disparity map and generating the first disparity map, such as Figure 4 shown.

[0114] Since the hole filling method adopted in this implementation is to obtain pixel values ​​from adjacent pixels, the filling effect is smoother.

[0115] It is worth noting that the four search directions of up, down, left and right are not fixed, and three directions of search, equal distance search, etc. can also be used. When the search encounters the boundary of the fourth disparity map and the adjacent pixels have not been found, the corresponding item can be deleted in the above formula.

[0116] Furthermore, before step B10, the method further includes:

[0117] Step C10: binarize the fourth disparity map to generate a binary disparity map.

[0118] Specifically, the fourth disparity map is first binarized, and the binarization can be achieved by conventional image threshold methods, such as threshold binarization, threshold debinarization, and truncation. The most common threshold binarization is described as an example. A certain value is set as the threshold, and if the pixel value is larger than the threshold, it is set to 1, and if it is smaller than the threshold, it is set to 0, so that the fourth disparity map is converted into a binary disparity map.

[0119] Step C20: determining a bright spot area of ​​the fourth disparity map according to the area and the mean value of the connected domain in the binary disparity map.

[0120] Specifically, first determine whether the area of ​​each connected domain is lower than a preset area threshold.

[0121] If so, the connected region is determined as a candidate region.

[0122] Calculate the pixel mean of each small connected domain and determine whether the pixel mean is greater than a preset pixel threshold

[0123] If so, the small connected domain is determined as a bright spot area.

[0124] There are two characteristics of bright spot areas: one is that they are concentrated in a small range, and the other is that the pixel value is relatively high. First, based on the area of ​​the connected domain, select the connected domain with a small distribution range as the candidate area; then, based on the mean of the pixel points, determine the bright spot area with a higher mean. It is worth noting that the basis for judgment is the characteristics of the bright spot area. There is no causal relationship between the two characteristics. Therefore, when judging, it is feasible to judge the area first and then the pixel value, or to judge the pixel value first and then the area.

[0125] Step C30: converting each pixel in the bright spot area into a hole pixel according to the hole pixel value.

[0126] Specifically, a preset hole pixel value is used to assign a value to each pixel in the bright spot area to generate a hole pixel, thereby completing the conversion of the bright spot area.

[0127] Therefore, in this embodiment, when performing hole filling, hole conversion is performed on the bright spot area, so that the holes and bright spots can be converted into effective pixels at the same time.

[0128] Step S200: generating a second disparity map corresponding to a preset target resolution according to the first disparity map.

[0129] Specifically, since the resolution of the first disparity map is low, in order to obtain a disparity map with a higher resolution, the resolution of the first disparity map needs to be adjusted. A target resolution is preset, and then the resolution of the first disparity map is adjusted to be the same as the target resolution to generate the second disparity map.

[0130] Furthermore, the resolution adjustment of the first disparity map may be implemented by adjusting the resolution of the depth map, and the specific process is as follows:

[0131] Step D10: generating a third depth map corresponding to the first disparity map according to preset camera parameters.

[0132] Specifically, the camera will be calibrated before leaving the factory, including internal parameters, camera optical center distance, etc. According to the parameters of the camera, the disparity map and the depth map can be converted, and the conversion formula is: C2 = (f*baseline) / C1, where C2 is the depth map, f is the internal parameter of the camera, baseline is the camera optical center distance, and C1 is the disparity map. Based on this formula and the preset camera parameters, the third depth map corresponding to the first disparity map can be generated.

[0133] Step D12, performing enhancement processing on the reference image to generate an intermediate image.

[0134] Specifically, see Figure 2 , taking image 1 as a reference image, performing enhancement processing on the reference image, such as color enhancement and noise reduction, so as to improve the color distinction of the reference image. The reference image can be a grayscale image or a color image. When the reference image is a grayscale image, performing noise reduction on the reference image to generate the intermediate image. The noise reduction processing method can adopt conventional Gaussian filtering, bilateral filtering, median filtering, etc. When the reference image is a color image, performing noise reduction on the reference image to generate a noise-reduced image; performing color enhancement on the noise-reduced image to generate an intermediate image.

[0135] Step D13: filtering the first disparity map according to the intermediate image to generate a fifth disparity map.

[0136] Specifically, the intermediate image is used as a guide map, and the first disparity map is used as a to-be-processed map, and filtering is performed to improve the smoothness of the disparity map. In order to reduce the loss of peripheral information of the image, the filtering method used is edge-preserving smoothing filtering, and the filtering algorithms that can be selected here include but are not limited to weighted least squares filtering, global smoothing filtering, and guide filtering, and finally a fifth disparity map is generated.

[0137] Step D14: generating a third depth map corresponding to the fifth disparity map according to the camera parameters.

[0138] Specifically, after the fifth disparity map is obtained according to the conversion formula between the depth map and the disparity map, and the camera parameters, the fifth disparity map can be converted into the third depth map.

[0139] Step D20: up-sample the third depth map according to a preset target resolution to generate a fourth depth map.

[0140] Specifically, the third depth map is first upsampled according to a preset target resolution to generate a fourth depth map. The upsampling method used may be selected but not limited to bilinear interpolation, cubic interpolation, nearest neighbor interpolation, etc. In this way, the resolution of the fourth depth map obtained is the same as the preset target resolution.

[0141] Step D30: converting the fourth depth map into a second disparity map according to the camera parameters.

[0142] Specifically, as known above, the depth map can be converted to the disparity map, so after obtaining the fourth depth map equal to the target resolution, the second depth map can be generated based on the conversion formula and the camera parameters.

[0143] Step S300: Processing a first depth map corresponding to the first disparity map according to the reference image to generate a second depth map.

[0144] Specifically, since the reference image and the first disparity map are both obtained by shooting the same scene, if the reference image is a high-resolution image, the reference image is directly used as a guide map to improve the resolution of the first depth map corresponding to the first disparity map to generate a second depth map. If the reference image is a low-resolution image, the resolution of the reference image is first improved, and then the reference image with the improved resolution is used as a guide map to improve the resolution of the first depth map corresponding to the first disparity map to generate a second depth map.

[0145] Furthermore, in order to make the second depth map with improved resolution more refined, this embodiment uses a mapping map and a confidence map to perform refinement. The specific process is:

[0146] Step E10: up-sample the intermediate image according to the target resolution to generate a target image.

[0147] Specifically, according to the target resolution, the intermediate image is upsampled to obtain an image with a resolution value equal to the target resolution, that is, the target image. The upsampling method has been described above and will not be repeated here.

[0148] Step E20, calculating the hash value corresponding to each target pixel in the target image, and using the coordinate value of the target pixel as an index value to generate a corresponding target hash value table.

[0149] Specifically, the hash value corresponding to each target pixel in the target image is calculated, that is, a hash value is used to perform feature synthesis on the target pixel. Then, each hash value is matched with the coordinate value of the target pixel to generate an initial hash value table. In this process, there may be conflicts, that is, the hash values ​​corresponding to multiple coordinate values ​​are the same. Therefore, in this process, the initial hash value table can be conflict-processed to generate a corresponding target hash value table.

[0150] Furthermore, step E20 includes:

[0151] Step F10, dividing the target image according to a preset division interval to generate a plurality of divided images.

[0152] Specifically, the target image is divided at a certain interval to generate a plurality of divided images. The division method may be selected but not limited to fixed interval division, and may also be equal division, random division, etc. Among them, fixed interval division is to set a fixed interval in advance and then divide the image at the interval; equal division is to set the number of divided images and then divide them equally according to the number.

[0153] Step F20 , calculating a hash value corresponding to each target pixel in the target image according to the division coordinates and division pixel values ​​of the plurality of division images.

[0154] Specifically, after the division, the coordinates and pixel values ​​of the divided image will change. Taking the interval division as an example, in this embodiment, the preset fixed interval is S, the pixel coordinates are (x1, y1), and then x2 in the divided coordinates after the division is taken as the integer part of x1 / S. In the same way, the divided coordinates (x2, y2) after the division are obtained. Because the disparity map and the depth map are both represented in the form of grayscale images, in order to better refine the disparity map, the pixel value of the target pixel is encoded in YUV. Pixels are also encoded in a similar manner. For example, the brightness Y1 corresponding to the target pixel value, Y2 takes the integer part of Y1 / S in the divided pixel value of the divided image after the division.

[0155] After obtaining the division coordinates and division pixel values ​​of each division pixel in the division image, the hash value corresponding to each target pixel is calculated by the following formula:

[0156] H i,j =x+255y+255 2 Y+255 3 U+255 4 V;

[0157] Among them, H i,j represents the hash value of the target pixel with coordinates (i, j) in the target image; x and y represent the horizontal and vertical coordinates of the target pixel with coordinates (i, j) in the target image in the split image; Y, U, V represent the split image corresponding to the target pixel with coordinates (i, j) in the target image. Since the color depth in the grayscale image is 0 to 255, 0 is black and 255 is white, the coefficient used to calculate the hash value is 255.

[0158] Step F30: associate the index value with the hash value, and create an initial hash table corresponding to the hash value.

[0159] Specifically, the coordinate value of the target vector is matched with the hash value to generate an initial hash table. For example, if the coordinate value of a target pixel is (1,1), the corresponding hash value is 123. Then the index value is (1,1), and the corresponding hash value 123 can be found in the initial hash value table.

[0160] In this embodiment, the target image is divided, and hash values ​​are calculated using the division coordinates and division pixel values ​​of the divided images, which can effectively reduce the probability of hash conflicts.

[0161] Step F40: updating the initial hash table according to the third depth map to generate a target hash table.

[0162] Specifically, since the third depth map is derived from the original image and the target image is also derived from the original image, some features of the target image should exist in the third depth map. Therefore, the hash value corresponding to the target image can be further processed according to the third depth map, so as to update the initial hash value table.

[0163] See also Figure 5First, since the third depth map is a low-resolution map, its edges are relatively messy. First, according to the third depth map and the target resolution, the support mask corresponding to the target image is calculated. The support mask refers to a mask generated according to the edge map of the third depth map. The specific generation process includes: performing edge extraction on the third depth map to generate a first edge map; performing upsampling on the first edge map according to the preset target resolution to generate a second edge map, such as Figure 2 As shown; according to a preset truncation threshold, each pixel value in the second edge map is threshold-truncated to generate a binary mask; the binary mask is morphologically expanded to generate a support mask corresponding to the target image.

[0164] Among them, the method of edge extraction for the third depth map includes but is not limited to the Sobel edge detection operator and the Laplacian edge detection in the discrete differentiation operator. The sampling method used for upsampling the first edge map to generate the second edge map can be the same as the sampling method used to generate the first depth map. The threshold truncation process is similar to the binarization process described above. Morphological dilation is a conventional method of image processing, which will not be described here one by one.

[0165] After obtaining the support mask, each target pixel of the target image is sampled and marked according to the support mask to generate sampled pixels and non-sampled pixels, wherein the non-sampled pixels are pixels in the target pixels that have not been sampled and marked. The sampling and marking process is: according to a preset high-density sampling value, the target pixels corresponding to the support mask in the target image are sampled and marked with high density, and according to a preset low-density sampling value, the target pixels corresponding to the support mask in the target image are sampled and marked with low density to generate sampled pixels. Among them, the sampling method can be selected but not limited to fixed interval sampling, fixed area sampling, etc., the preset high-density sampling value and the low-density sampling value can be adjusted according to actual conditions, such as high density 1:3 and low density 1:6.

[0166] Finally, the initial hash table is updated according to the coordinate value and the corresponding hash value of the sampled pixel to generate the target hash table. The specific process includes: calculating the sampling hash value corresponding to the sampled pixel; traversing the sampled pixels and determining whether there is a hash value in the initial hash table that is the same as the sampling hash value corresponding to the sampled pixel; if not, adding the coordinate value and the corresponding sampling hash value of the sampled pixel to the initial hash table to generate the target hash table.

[0167] Specifically, for example, in the initial hash table, the hash values ​​corresponding to the index values ​​(1,1) and (4,4) are both 123. In this round, the target pixel with the coordinate value (4,4) is the sampling pixel, because the hash value calculation this time is no longer based on the segmented image, but based on the target image, so the probability that the hash values ​​of the two are the same is very low, and the corresponding sampling hash value 233 is calculated based on the coordinates and pixel value of the target pixel. Then the index value (4,4) and the corresponding hash value 233 are added to the initial hash table to generate the target hash table.

[0168] In this embodiment, when the division interval selected when calculating the bilateral domain information is greater than 1, there is a situation where the coordinate values ​​and pixels corresponding to different pixels are the same after division, then the calculated hash values ​​will be the same, and the two pixels cannot be distinguished in the bilateral domain. The more pixels that cannot be distinguished like this, the greater the compression ratio of the bilateral domain, and the smaller the corresponding disparity map filtering calculation amount. Therefore, this embodiment performs sparse mapping by masking on the basis of image division, thereby updating the initial hash table, thereby ensuring that the hash values ​​corresponding to different target pixels are different.

[0169] Step E30: Calculate a mapping map corresponding to the target image according to the target hash value table.

[0170] Specifically, after the hash value corresponding to each target pixel point is obtained, the hash values ​​corresponding to each target pixel value in the target image are mapped, thereby generating a corresponding mapping graph.

[0171] Furthermore, since the hash value corresponding to each target pixel is calculated in the form of bilateral hashing, and the hash value is generally long, the data volume is too large if the hash value is directly used for calculation. In order to reduce the amount of calculation, this embodiment provides a mapping table with index values ​​as objects, so step E30 includes:

[0172] Step E31: when the target pixel is a sampling pixel, according to the target hash table, the index value corresponding to the sampling pixel is used as the mapping value corresponding to the sampling pixel.

[0173] Specifically, when the target pixel is a sampled pixel, its corresponding hash value can definitely be found in the target hash table, so the index value in the corresponding target hash table is returned as a mapping result of the bilateral hash.

[0174] Step E32: when the target pixel is a non-sampled pixel, determining a mapping value corresponding to the non-sampled pixel according to whether there is a hash value corresponding to the non-sampled pixel in the target hash table.

[0175] Specifically, when the target pixel is a non-sampled pixel, its hash value is calculated to generate a non-sampled hash value. It is determined whether there is a hash value equal to the non-sampled hash value in the target hash table. If there is, the hash value can be directly used as the hash value corresponding to the non-sampled pixel. This process can be specifically as follows:

[0176] For each non-sampled pixel, when the target hash table has a hash value corresponding to the non-sampled pixel, calculate an index value corresponding to the hash value, and use the index value as a mapping value corresponding to the non-sampled pixel;

[0177] When the target hash table does not contain the hash value corresponding to the non-sampled pixel, a preset null value is used as the mapping value corresponding to the non-sampled pixel.

[0178] Specifically, if the non-sampled hash value can be found in the target hash table, then the index value in the corresponding target hash table is returned as the mapping result of the bilateral hash of the non-sampled pixel. If it does not exist, a preset null value is used as the mapping value of the non-sampled pixel. In this embodiment, it is represented by a negative value.

[0179] Step E33: generating a mapping image corresponding to the target image according to the coordinates of the sampled pixels, the coordinates of the non-sampled pixels and the mapping values.

[0180] Specifically, the coordinates of the sampled pixels and the non-sampled pixels are finally written with the corresponding mapping values ​​into a preset blank matrix to generate a mapping diagram corresponding to the target image. The blank matrix is ​​created in advance according to the target resolution. Therefore, this embodiment combines the above two points, and while greatly compressing the length of the final hash table, it can also effectively ensure that there is no conflict between the index value and the hash value, thereby significantly reducing the amount of calculation and ensuring the accuracy of the calculation.

[0181] Step E40: Calculate a confidence map corresponding to the first depth map, and calculate a second depth map corresponding to the mapping map based on the confidence map and the first depth map.

[0182] Specifically, the confidence map corresponding to the first depth map refers to a map composed of the credibility of pixel values ​​in the first depth map. The confidence map corresponding to the first depth map is first calculated, and then a bilateral solution is used to combine the confidence map and the first depth map to calculate the second depth map corresponding to the mapping map.

[0183] Further, step E40 includes:

[0184] Step E41, inverting the second edge map to generate a confidence map corresponding to the first depth map, and performing a bilateral solution on the mapping map based on the confidence map and the first depth map to generate depth values ​​corresponding to each index value in the target hash table.

[0185] Specifically, since the second edge map is obtained by edge extraction of the first depth map, and edge information is most easily distorted subsequently, the image obtained by inverting the second edge map can be regarded as a confidence map. Since the pixel value range in the grayscale image is 0 to 255, the calculation formula of the confidence map is F2=255-E2, where E2 is the second edge map and F2 is the confidence map.

[0186] Then, according to the confidence map and the first depth map, the mapping map is bilaterally solved to generate depth values ​​corresponding to each index value in the target hash table. Bilateral solution is an iterative solution method based on global bilateral constraints. Since bilateral solution is a prior art, its process will not be described one by one here.

[0187] Step E42, generating a fifth depth map corresponding to the target pixel according to the mapping value in the mapping map and the depth value corresponding to the index value, wherein when the mapping value is a null value, the pixel value corresponding to the target pixel in the fifth depth map is a null value.

[0188] Specifically, a table lookup method is used to determine the depth value corresponding to the target pixel. Since the target image has the same resolution as the mapping map, each target pixel can find a corresponding mapping value in the mapping map. Since the mapping values ​​in the mapping map include two types, one is to directly use the index value as the mapping value, and the other is a null value as the mapping value. Therefore, when the mapping value is an index value, that is, a non-negative value, the corresponding depth value is determined according to the index value to be the depth value corresponding to the target pixel. When the mapping value is a null value, that is, a negative value, the depth value corresponding to the target pixel is null, and in this embodiment, it is a negative value. The depth value corresponding to the target pixel is written into a preset blank matrix to generate the fifth depth map. Since the number of sampled target pixels is not large, a large number of values ​​in the mapping map are null values, so the fifth depth map is a sparse depth map.

[0189] Step E43: according to the target image, depth pixels whose pixel values ​​are null values ​​in the fifth depth map are taken as hole pixels, and the holes are filled to generate a sixth depth map.

[0190] Specifically, a hole filling method similar to the above-mentioned method for filling holes in the fourth disparity map is adopted, and depth pixels whose pixel values ​​are null values ​​are used as hole pixels to fill holes, thereby generating a sixth depth map. Since a large number of null values ​​in the fifth depth map are filled, the obtained sixth depth map is a dense depth map.

[0191] Step E44: compare the pixel values ​​of the same pixel in the first depth map and the sixth depth map, and update the pixel value of the sixth depth map according to the larger pixel value to generate a second depth map.

[0192] Specifically, the values ​​of the same pixel in the first depth map and the sixth depth map are compared at the pixel level, and the larger value is used as the value in the sixth depth map. For example, the pixel with coordinates (1, 1) has a value of 4 in the first depth map and 2 in the sixth depth map. Therefore, the pixel value with coordinates (1, 1) in the sixth depth map is updated to 4. After completing the update of all pixels, the second depth map is output.

[0193] Step S400: determining a third disparity map according to the second depth map and the second disparity map.

[0194] Specifically, since the second depth map has a higher resolution and is more detailed, the second depth map may be combined with the second disparity map to determine a third disparity map with a higher level of detail.

[0195] Furthermore, the second depth map and the second disparity map may be combined by processing the second disparity map according to the second depth map to obtain a third disparity map. Figure 6 As shown, the second depth map can be used as a guide map to filter and correct the second disparity map to generate a smoother third disparity map. The preferred processing method used in this embodiment is filtering, such as edge-preserving smoothing filtering.

[0196] Furthermore, after the third disparity map is obtained, three-dimensional reconstruction is performed according to the third disparity map, the reference image and the original image to obtain a three-dimensional scene corresponding to the target scene.

[0197] Specifically, since disparity maps and depth maps can record the distance between objects in the scene and the camera, they can be used for object measurement, 3D reconstruction, and virtual viewpoints. Taking 3D reconstruction as an example, first, the corresponding target depth map is obtained based on the third disparity map. The target depth map contains depth information (Z c), and then construct a three-dimensional point cloud according to the coordinates of the pixel points corresponding to each pixel in the reference image and the original image, as well as the pixel value of each pixel point, so as to realize the three-dimensional reconstruction of the target scene and obtain the corresponding three-dimensional scene.

[0198] In terms of virtual viewpoints, the third disparity map can be used directly, according to the coordinates and pixels corresponding to each pixel in the reference image or the original image, to obtain a virtual viewpoint position image corresponding to the target scene.

[0199] In addition, the third disparity map can be converted into a depth map based on a conversion formula based on the disparity map and the depth map to obtain a target depth map, and then the holes of the original image or the reference image can be filled based on the intrinsic parameter matrix of the camera and the target depth map to obtain a virtual viewpoint position image.

[0200] Furthermore, the parameters of the intelligent terminal used in this embodiment are as follows:

[0201] Phone model Alcatel Venice CPU MediaTek TM6577 Rear camera Wide-angle dual-camera, no bracket required Shooting environment illumination range >100Lux Programming Language Native C++ (developed based on Android NDK) Operational efficiency 816x612 resolution super resolution takes only 350ms

[0202] Experiments have shown that conventional bilateral solution takes more than 2 seconds, while this embodiment only takes 350 milliseconds. Therefore, this embodiment can greatly improve the speed of image processing.

[0203] Furthermore, if Figure 7 As shown, based on the above image processing method, the present invention also provides a smart terminal, which includes a processor 10, a memory 20 and a display 30. Figure 7 Only some components of the smart terminal are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0204] In some embodiments, the memory 20 may be an internal storage unit of the smart terminal, such as a hard disk or memory of the smart terminal. In other embodiments, the memory 20 may also be an external storage device of the smart terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the smart terminal. Further, the memory 20 may also include both an internal storage unit of the smart terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the smart terminal, such as the program code of the installed smart terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, an image processing program 40 is stored on the memory 20, and the image processing program 40 can be executed by the processor 10, thereby realizing the image processing method in the present application.

[0205] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the image processing method.

[0206] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the smart terminal and to display a visual user interface. The components 10-30 of the smart terminal communicate with each other via a system bus.

[0207] In one embodiment, the above-mentioned image processing method is implemented when the processor 10 executes the image processing program 40 in the memory 20 .

[0208] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method described above are implemented.

[0209] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0210] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Providing a reference image and an original image for the same target scene, and performing disparity processing on the original image to generate a corresponding first disparity map; Generating a second disparity map corresponding to a preset target resolution according to the first disparity map; Processing, according to the reference image, a resolution of a first depth map corresponding to the first disparity map to generate a second depth map, wherein the resolution of the second depth map is higher than the resolution of the first depth map; A third disparity map is determined according to the second depth map and the second disparity map.

2. The image processing method according to claim 1, characterized in that: The providing of a reference image and an original image for the same target scene specifically includes: Controlling a preset binocular camera to shoot the target scene to obtain a binocular image, wherein the binocular image includes a left-eye image and a right-eye image; The left eye image or the right eye image is selected as a reference image, and the other binocular image is used as an original image.

3. The image processing method according to claim 1, characterized in that: The performing disparity processing on the original image to generate a corresponding first disparity map specifically includes: According to the reference image, performing disparity calculation on the original image to generate a corresponding fourth disparity map; Holes are filled in the fourth disparity image to generate a first disparity image corresponding to the fourth disparity image.

4. The image processing method according to claim 3, characterized in that: The filling of holes in the fourth disparity map to generate a first disparity map corresponding to the fourth disparity map specifically includes: Filtering hole pixels in the fourth disparity map according to a preset hole pixel value; Determine corresponding neighboring pixels according to the coordinates of the hole pixel, wherein the neighboring pixels are pixels that are adjacent to the hole pixel and are not hole pixels; Calculating a filling value corresponding to the hole pixel according to the hole pixel and the neighboring pixels; According to the filling value, pixel values ​​of the hole pixels are assigned to generate a first disparity map.

5. The image processing method according to claim 4, characterized in that: Before screening the hole pixels in the fourth disparity map according to the preset hole pixel values, the method further includes: Binarizing the fourth disparity map to generate a binary disparity map; determining a bright spot area of ​​the fourth disparity map according to an area and a mean value of a connected domain in the binary disparity map; According to the hole pixel value, each pixel in the bright spot area is converted into a hole pixel.

6. The image processing method according to claim 1, characterized in that: Generating a second disparity map corresponding to a preset target resolution according to the first disparity map specifically includes: Generating a third depth map corresponding to the first disparity map according to preset camera parameters; Upsampling the third depth map according to a preset target resolution to generate a fourth depth map; The fourth depth map is converted into a second disparity map according to preset camera parameters.

7. The image processing method according to claim 6, characterized in that: The step of generating a third depth map corresponding to the first disparity map according to preset camera parameters specifically includes: Performing enhancement processing on the reference image to generate an intermediate image; performing filtering processing on the first disparity map according to the intermediate image to generate a fifth disparity map; A third depth map corresponding to the fifth disparity map is generated according to the camera parameters.

8. The image processing method according to claim 7, characterized in that: The reference image includes a grayscale image and a color image; and the enhancing process on the reference image to generate an intermediate image specifically includes: When the reference image is a grayscale image, performing noise reduction processing on the reference image to generate the intermediate image; When the reference image is a color image, noise reduction processing is performed on the reference image to generate a noise-reduced image, and color enhancement is performed on the noise-reduced image to generate an intermediate image.

9. The image processing method according to claim 7 or 8, characterized in that: The processing, according to the reference image, of a first depth map corresponding to the first disparity map to generate a second depth map specifically includes: Upsampling the intermediate image according to the target resolution to generate a target image; Calculate the hash value corresponding to each target pixel in the target image, and use the coordinate value of the target pixel as an index value to generate a corresponding target hash value table; Calculating a mapping map corresponding to the target image according to the target hash value table; A confidence map corresponding to the first depth map is calculated, and a second depth map corresponding to the mapping map is calculated based on the confidence map and the first depth map.

10. The image processing method according to claim 9, characterized in that: The step of calculating the hash value corresponding to each target pixel in the target image and using the coordinate value of the target pixel as an index value to generate a corresponding target hash value table specifically includes: Dividing the target image according to a preset division interval to generate a plurality of divided images; Calculating a hash value corresponding to each target pixel in the target image according to the division coordinates and the division pixel values ​​of the plurality of division images; Matching the index value with the hash value, and creating an initial hash table corresponding to the hash value; The initial hash table is updated according to the third depth map to generate a target hash table.

11. The image processing method according to claim 10, characterized in that: The updating the initial hash table according to the third depth map to generate a target hash table specifically includes: Calculating a support mask corresponding to the target image according to the third depth map and the target resolution; According to the support mask, each target pixel of the target image is sampled and marked to generate sampled pixels and non-sampled pixels, wherein the non-sampled pixels are pixels that are not sampled and marked among the target pixels; The initial hash table is updated according to the coordinate values ​​of the sampled pixels and the corresponding hash values ​​to generate a target hash table.

12. The image processing method according to claim 11, characterized in that: The calculating, according to the third depth map and the target resolution, a support mask corresponding to the target image specifically includes: Performing edge extraction on the third depth map to generate a first edge map; Upsampling the first edge map according to a preset target resolution to generate a second edge map; According to a preset truncation threshold, threshold truncation is performed on each pixel value in the second edge image to generate a binary mask; The binary mask is morphologically expanded to generate a support mask corresponding to the target image.

13. The image processing method according to claim 11, characterized in that: The step of performing sampling marking on each target pixel of the target image according to the support mask to generate sampled pixels and non-sampled pixels specifically includes: According to a preset high-density sampling value, a target pixel in the target image corresponding to the support mask is sampled and marked, and according to a preset low-density sampling value, a target pixel in the target image corresponding to the support mask is sampled and marked to generate a sampling pixel; Pixels in the target pixels that are not marked as sampling pixels are regarded as non-sampling pixels.

14. The image processing method according to claim 11, characterized in that: The updating of the initial hash table according to the coordinate values ​​of the sampled pixels and the corresponding hash values ​​to generate a target hash table specifically includes: Calculating a sampling hash value corresponding to the sampled pixel; Traversing the sampled pixels, and determining whether there exists in the initial hash table a hash value identical to the sampled hash value corresponding to the sampled pixel; If not, the coordinate value of the sampled pixel and the corresponding sampled hash value are added to the initial hash table to generate the target hash table.

15. The image processing method according to claim 11, characterized in that: The calculating, according to the target hash value table, a mapping map corresponding to the target image specifically includes: When the target pixel is a sampling pixel, according to the target hash table, the index value corresponding to the sampling pixel is used as the mapping value corresponding to the sampling pixel; When the target pixel is a non-sampled pixel, determining a mapping value corresponding to the non-sampled pixel according to whether a hash value corresponding to the non-sampled pixel exists in the target hash table; A mapping image corresponding to the target image is generated according to the coordinates of the sampled pixels, the coordinates of the non-sampled pixels and the mapping values.

16. The image processing method according to claim 15, characterized in that: The determining, according to whether there is a hash value corresponding to the non-sampled hash value corresponding to the non-sampled pixel in the target hash table, a mapping value corresponding to the non-sampled pixel specifically includes: For each non-sampled pixel, when the target hash table has a hash value corresponding to the non-sampled pixel, calculate an index value corresponding to the hash value, and use the index value as a mapping value corresponding to the non-sampled pixel; When the target hash table does not contain the hash value corresponding to the non-sampled pixel, a preset null value is used as the mapping value corresponding to the non-sampled pixel.

17. The image processing method according to claim 12, characterized in that: The calculating a confidence map corresponding to the first depth map, and calculating a second depth map corresponding to the mapping map according to the confidence map and the first depth map, specifically includes: Inverting the second edge map to generate a confidence map corresponding to the first depth map, and performing a bilateral solution on the mapping map according to the confidence map and the first depth map to generate depth values ​​corresponding to each index value in the target hash table; Generate a fifth depth map corresponding to the target pixel according to the mapping value in the mapping map and the depth value corresponding to the index value, wherein when the mapping value is a null value, the pixel value corresponding to the target pixel in the fifth depth map is a null value; According to the target image, depth pixels whose pixel values ​​in the fifth depth map are null values ​​are used as hole pixels, and the holes are filled to generate a sixth depth map; The pixel value of the sixth depth map is updated according to the updated pixel value to generate a second depth map, wherein the updated pixel value is a maximum value of pixel values ​​in the first depth map and the sixth depth map when corresponding to the same pixel.

18. The image processing method according to claim 1, characterized in that: The determining the third disparity map according to the second depth map and the second disparity map specifically includes: The second disparity map is processed according to the second depth map to obtain a third disparity map.

19. The image processing method according to claim 18, characterized in that: The processing of the second disparity map according to the second depth map to obtain a third disparity map specifically includes: According to the second depth map, smoothing filtering is performed on the second disparity map to obtain a third disparity map.

20. The image processing method according to claim 1, characterized in that: After determining the third disparity map according to the second depth map and the second disparity map, the method further includes: Three-dimensional reconstruction is performed according to the third disparity map, the reference image and / or the original image to obtain a three-dimensional scene corresponding to the target scene.

21. An intelligent terminal, characterized in that: The intelligent terminal comprises: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program implements the steps of the image processing method according to any one of claims 1 to 20 when executed by the processor.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an image processing program, and when the image processing program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 20 are implemented.

Images