Image processing methods, apparatuses, electronic devices and storage media

Abstract

This application proposes an image processing method, apparatus, electronic device, and storage medium. The method includes: acquiring a first color image and a second color image to be processed; wherein the first color image and the second color image are images of the same scene from different angles; extracting image contours from the first color image and the second color image respectively to obtain a first contour map and a second contour map; acquiring a color contour image based on the first color image and the first contour map; calculating the disparity of the region containing the contour based on the color contour image and the second color image, and comparing and verifying the disparity of the region containing the contour with the first contour map and the second contour map to generate a disparity map of the contour; and generating a depth map of the contour based on the disparity map of the contour. This application can solve the problems of existing depth calculations, such as requiring large amounts of computing resources, slow calculation speed, large calculation errors, and poor results.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Binocular stereo vision is an important form of machine vision. It's based on the principle of parallax and uses an imaging device to acquire two images of a test object from different positions. By calculating the positional deviation between corresponding points in the images, it obtains the three-dimensional geometric information of the object. Related technologies typically employ full-field computation, traversing every pixel in both images of the test object for calculation. However, this depth calculation method, due to its full-field computation, consumes significant computational resources and is slow. Furthermore, this method performs poorly in areas with weak texture, resulting in large calculation errors and significant noise in the resulting depth map. Summary of the Invention

[0003] The purpose of this application is to at least partially solve one of the aforementioned technical problems.

[0004] Therefore, the first objective of this application is to propose an image processing method to solve the problems of existing technologies, such as the high computational resource consumption and slow computation speed of depth calculation.

[0005] The second objective of this application is to provide an image processing apparatus.

[0006] The third objective of this application is to propose an electronic device.

[0007] The fourth objective of this application is to provide a computer-readable storage medium.

[0008] The fifth objective of this application is to provide a computer program product.

[0009] To achieve the above objectives, a first aspect of this application provides an image processing method, comprising:

[0010] Acquire a first color image and a second color image to be processed; wherein the first color image and the second color image are images of the same scene from different angles;

[0011] Image contour extraction is performed on the first color image to obtain a first contour map;

[0012] A color contour image is obtained based on the first color image and the first contour image; wherein, the color contour image is the image after removing the non-contour regions from the first color image;

[0013] The disparity of the region containing the contour is calculated based on the color contour image and the second color image, and a disparity map of the contour is generated based on the disparity of the region containing the contour; and

[0014] A depth map of the contour is generated based on the disparity map of the contour.

[0015] In some embodiments, the first color image and the second color image are respectively a first image and a second image captured by a binocular camera.

[0016] In some embodiments, obtaining the color contour image based on the first color image and the first contour image includes:

[0017] The pixel matrices corresponding to the first color image and the first contour image are multiplied by a dot product operation to obtain the color contour image.

[0018] In some embodiments, before obtaining the color contour image based on the first color image and the first contour map, the method further includes:

[0019] The first contour image is preprocessed; wherein the preprocessing includes at least one or more of the following: denoising, binarization, and morphological dilation.

[0020] The step of obtaining a color contour image based on the first color image and the first contour image includes:

[0021] The color contour image is obtained based on the first color image and the preprocessed first contour image.

[0022] In some embodiments, calculating the disparity of the region containing the contour based on the color contour image and the second color image includes:

[0023] The disparity of the region containing the contour is obtained by calculating the disparity between the pixels corresponding to the contour position in the color contour image and the pixels corresponding to the same position in the second color image.

[0024] In some embodiments, the method further includes:

[0025] Image contour extraction is performed on the second color image to obtain a second contour map;

[0026] The step of generating a disparity map of the contour based on the disparity of the region where the contour is located includes:

[0027] The disparity of the region containing the contour is compared and verified with the first contour map and the second contour map to generate a disparity map of the contour.

[0028] In this embodiment of the application, the step of comparing and verifying the disparity of the region where the contour is located with the first contour map and the second contour map to generate a disparity map of the contour includes:

[0029] Traverse each pixel in the first contour map and obtain the disparity corresponding to the current pixel from the disparity of the region where the contour is located.

[0030] The coordinates of the current pixel are added to the disparity corresponding to the current pixel, and it is determined whether the coordinates obtained after the addition are on the contour position in the second contour map.

[0031] If so, retain the disparity corresponding to the current pixel; otherwise, delete the disparity corresponding to the current pixel.

[0032] Until all pixels in the first contour map have been traversed, the remaining disparity in the region where the contour is located is used as the disparity map of the contour.

[0033] To achieve the above objectives, a second aspect of this application provides an image processing apparatus, comprising:

[0034] The first acquisition module is used to acquire a first color image and a second color image to be processed; wherein the first color image and the second color image are images from the same scene;

[0035] The contour extraction module is used to extract the image contour from the first color image to obtain a first contour map.

[0036] The second acquisition module is used to acquire a color contour image based on the first color image and the first contour image; wherein the color contour image is the image after removing non-contour regions from the first color image;

[0037] A disparity map generation module is used to calculate the disparity of the region containing the contour based on the color contour image and the second color image, and to generate a disparity map of the contour based on the disparity of the region containing the contour; and

[0038] A depth map generation module is used to generate a depth map of the contour based on the disparity map of the contour.

[0039] In some embodiments, the first color image and the second color image are respectively a first image and a second image captured by a binocular camera.

[0040] In some embodiments, the second acquisition module is specifically used for:

[0041] The pixel matrices corresponding to the first color image and the first contour image are multiplied by a dot product operation to obtain the color contour image.

[0042] In some embodiments, the apparatus further includes:

[0043] A preprocessing module is used to preprocess the first contour image; wherein the preprocessing includes at least one or more of the following: denoising, binarization, and morphological dilation.

[0044] In this embodiment, the second acquisition module is specifically used to: acquire the color contour image based on the first color image and the preprocessed first contour image.

[0045] In some embodiments, the disparity map generation module is specifically used for:

[0046] The disparity of the region containing the contour is obtained by calculating the disparity between the pixels corresponding to the contour position in the color contour image and the pixels corresponding to the same position in the second color image.

[0047] In some embodiments, the contour extraction module is further configured to extract image contours from the second color image to obtain a second contour map;

[0048] The disparity map generation module is specifically used for:

[0049] The disparity of the region containing the contour is compared and verified with the first contour map and the second contour map to generate a disparity map of the contour.

[0050] In this embodiment of the application, the disparity map generation module is specifically used for:

[0051] Traverse each pixel in the first contour map and obtain the disparity corresponding to the current pixel from the disparity of the region where the contour is located.

[0052] The coordinates of the current pixel are added to the disparity corresponding to the current pixel, and it is determined whether the coordinates obtained after the addition are on the contour position in the second contour map.

[0053] If so, retain the disparity corresponding to the current pixel; otherwise, delete the disparity corresponding to the current pixel.

[0054] Until all pixels in the first contour map have been traversed, the remaining disparity in the region where the contour is located is used as the disparity map of the contour.

[0055] To achieve the above objectives, a third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image processing method described in the first aspect of this application.

[0056] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method described in the first aspect of this application.

[0057] To achieve the above objectives, a fifth aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the image processing method described in the first aspect of this application.

[0058] According to the technical solution of this application embodiment, a first color image and a second color image to be processed are acquired, and image contours are extracted from the first color image and the second color image respectively to obtain a first contour map and a second contour map. A color contour image is obtained based on the first color image and the first contour map, and the disparity of the region where the contour is located is calculated based on the color contour image and the second color image. The disparity of the region where the contour is located is compared and verified with the first contour map and the second contour map to generate a disparity map of the contour. Based on the disparity map of the contour, a depth map of the contour is generated. Therefore, this application extracts contours from two images used for depth calculation. Since the target in the field of view usually has a large grayscale gradient with the background, a very obvious contour image can be extracted. Thus, full-field calculation is not required; only the depth map of the contour region is calculated, which can save a significant amount of computational resources and improve the calculation speed. Furthermore, by comparing and verifying the contour maps of the two images with the disparity of the region where the contour is located, effective disparity can be extracted from the disparity of the region where the contour is located. Therefore, the error obtained through the double verification method of contour and disparity comparison can reduce the error of disparity calculation using a single calculation method, improving the correctness and effectiveness of disparity calculation.

[0059] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0060] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0061] Figure 1 A flowchart of an image processing method according to an embodiment of this application.

[0062] Figure 2 This is a flowchart of an image processing method according to another embodiment of this application.

[0063] Figure 3 This is a schematic flowchart of an image processing method according to an embodiment of this application. Figure 4 This is a structural block diagram of an image processing apparatus according to an embodiment of the present application.

[0064] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0065] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0066] The image processing method, apparatus, electronic device, and storage medium of this application are described below with reference to the accompanying drawings.

[0067] Figure 1 A flowchart of an image processing method according to an embodiment of this application. It should be noted that the image processing method of this application embodiment is applied to the image processing apparatus of this application embodiment, which is configured on an electronic device. Figure 1 As shown, the image processing method may include the following steps.

[0068] In step 101, a first color image and a second color image to be processed are obtained; wherein the first color image and the second color image are images of the same scene from different angles.

[0069] In some embodiments, the first color image and the second color image are respectively a first image and a second image captured by a stereo camera. For example, the first color image may be a left image captured by the stereo camera, and the second color image may be a right image captured by the stereo camera; or, for example, the first color image may be a right image captured by the stereo camera, and the second color image may be a left image captured by the stereo camera. As an example, the first color image and the second color image may be RGB images respectively.

[0070] In other embodiments, the first color image and the second color image can be two images taken by a monocular camera at different angles, wherein the first color image and the second color image must be images of the same scene from different angles. For example, the first color image and the second color image can also be two images taken by a monocular camera at different angles in the same scene.

[0071] In step 102, the first color image is subjected to image contour extraction to obtain a first contour map.

[0072] Optionally, the first color image can be converted into a grayscale image. For example, if the first color image is an RGB image, the 8-bit RGB image corresponding to the first color image can be converted into the corresponding grayscale image using the following conversion formula (1). When obtaining the grayscale image corresponding to the first color image, a contour extraction operator can be used to extract the contours in the grayscale image corresponding to the first color image to obtain the corresponding first contour image.

[0073] f(x)=R*0.3+G*0.51+B*0.11 (1)

[0074] Where f(x) is the gray value.

[0075] In step 103, a color contour image is obtained based on the first color image and the first contour image; wherein, the color contour image is the image after removing the non-contour regions from the first color image.

[0076] It can be understood that an image is essentially a pixel matrix. A color contour image can be obtained by multiplying the corresponding pixels of the first color image and the first contour image. Optionally, in some embodiments, the pixel matrices corresponding to the first color image and the first contour image are multiplied by a dot product operation to obtain the color contour image. Therefore, by multiplying the first color image with its corresponding contour image, non-contour regions in the first color image can be removed, retaining only the contour regions. This allows for subsequent disparity calculations using only the contour regions of the first color image with the second color image. This eliminates the need for full-field calculations on the color image, significantly saving computational resources and avoiding wasted computational resources and slow processing speeds caused by invalid calculations.

[0077] In step 104, the disparity of the region containing the contour is calculated based on the color contour image and the second color image, and a disparity map of the contour is generated based on the disparity of the region containing the contour.

[0078] Optionally, the disparity of the region containing the contour is obtained by calculating the disparity between the pixels corresponding to the contour position in the color contour image and the pixels corresponding to the same position in the second color image.

[0079] In some embodiments, image correlation matching can be used to calculate the disparity between pixels corresponding to contour positions in a color contour image and pixels at the same positions in a second color image. It should be noted that there are many image matching algorithms, such as Mean Absolute Difference (MAD), Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), and Normalized Cross-Correlation (NCC). This application does not specifically limit the image matching algorithm used.

[0080] For example, image correlation matching can be used to find matching point pairs between the contour position in the color contour image and the second color image. The disparity between the two points can be calculated using the coordinates of the two pixels in each matching point pair, thereby revealing the disparity of the region where the contour is located.

[0081] In this step, after obtaining the disparity of the region where the contour is located, the image composed of the disparity of the region where the contour is located can be used as the disparity map of the contour.

[0082] To ensure computational accuracy and improve the correctness and effectiveness of disparity calculation, optionally, in some embodiments of this application, image contour extraction can be performed on the second color image to obtain a second contour map. After calculating the disparity of the region containing the contour based on the color contour image and the second color image, the disparity of this region can be compared and verified with the first and second contour maps to generate a disparity map of the contour. In this application, the purpose of comparing and verifying the disparity of the region containing the contour with the first and second contour maps is to extract the effective disparity from the disparity of the region containing the contour. Therefore, the error obtained through the dual verification method of contour and disparity comparison can reduce the error of disparity calculation using a single calculation method, thereby improving the correctness and effectiveness of disparity calculation.

[0083] In step 105, a depth map of the contour is generated based on the disparity map of the contour.

[0084] Optionally, when obtaining the disparity map of the contour, the depth information of each contour pixel in the contour map can be calculated using each disparity value in the disparity map according to the depth calculation formula.

[0085] According to the image processing method of this application embodiment, a first color image and a second color image to be processed are acquired, and image contours are extracted from the first color image and the second color image respectively to obtain a first contour map and a second contour map. A color contour image is obtained based on the first color image and the first contour map, and the disparity of the region containing the contour is calculated based on the color contour image and the second color image. The disparity of the region containing the contour is compared and verified with the first contour map and the second contour map to generate a disparity map of the contour. A depth map of the contour is then generated based on the disparity map. Therefore, this application extracts contours from two images used for depth calculation. Since the target in the field of view usually has a large gray-level gradient with the background, a very obvious contour image can be extracted. Thus, full-field calculation is not required; only the depth map of the contour region is calculated, which can save a significant amount of computational resources and improve the calculation speed. Furthermore, by comparing and verifying the contour maps of the two images with the disparity of the region containing the contour, effective disparity can be extracted from the disparity of the region containing the contour. Therefore, the error obtained through the dual verification method of contour and disparity comparison can reduce the error of disparity calculation using a single calculation method, improving the accuracy and effectiveness of disparity calculation.

[0086] Figure 2 This is a flowchart of an image processing method according to another embodiment of this application. Figure 3 This is a schematic flowchart of an image processing method according to an embodiment of this application. In this embodiment, the description is based on the example of a first color image and a second color image being a left and right image captured by a binocular camera, respectively. Figure 2 and Figure 3 As shown, the image processing method may include the following steps.

[0087] In step 201, the left and right images captured by the binocular camera are acquired.

[0088] As an example, the left and right images can be RGB images, respectively.

[0089] In step 202, image contour extraction is performed on the left image to obtain a left contour map, and image contour extraction is performed on the right image to obtain a right contour map.

[0090] Optionally, the left and right images can be converted into grayscale images respectively. For example, if the left and right images are RGB images respectively, the 8-bit RGB image corresponding to the left image can be converted into the corresponding grayscale image using the following conversion formula (1), and the 8-bit RGB image corresponding to the right image can be converted into the corresponding grayscale image. When obtaining the grayscale images corresponding to the left and right images respectively, a contour extraction operator can be used to extract the contour in the grayscale image corresponding to the left image to obtain the corresponding left contour image, and a contour extraction operator can be used to extract the contour in the grayscale image corresponding to the right image to obtain the corresponding right contour image.

[0091] f(x)=R*0.3+G*0.51+B*0.11 (1)

[0092] Where f(x) is the gray value.

[0093] It should be noted that in some embodiments, the contour extraction operator can be the discrete differential Sobel operator. In other embodiments, the contour extraction operator can also be other operators, such as the Prewitt operator, the Roberts operator, and the Canny method of multi-level edge detection algorithms, which can also be used for contour extraction.

[0094] In step 203, the left and right contour images are preprocessed respectively.

[0095] In some embodiments, the preprocessing includes, but is not limited to, any one or more of the following: denoising, binarization, morphological dilation.

[0096] As an example, the preprocessing may include denoising, binarization, and morphological dilation. That is, the left contour image may be denoised, binarized, and morphologically dilated, and the right contour image may also be denoised, binarized, and morphologically dilated.

[0097] In this embodiment, median filtering or mean filtering can be used to remove salt-and-pepper noise from the left and right contour images. As an example, the template size used for filtering can be selected as 3×3, 5×5, or 7×7 depending on the actual effect. This effectively eliminates the interference of salt-and-pepper noise.

[0098] Optionally, in this embodiment, the left and right contour images can be binarized separately. The binarized value of contoured regions is 1, and the binarized value of non-contoured regions is 0. The binarization threshold can be between 0.3 and 0.7. Therefore, by using the binarized left contour image as a mask to perform matching calculations on the right image, weak texture areas in the image can be effectively excluded from the calculation range. Irrelevant weak texture areas will not be reflected in the contour image, effectively avoiding the influence of noisy weak texture areas on depth calculation.

[0099] In this embodiment, by performing morphological dilation on the left and right contour images respectively, the contours in the left and right contour images can be thickened. The kernel of the dilation operation can be 3×3, or other kernel sizes such as 5×5 or 7×7 can also be selected. Thus, by performing morphological dilation on the left and right contour images, the contour area can be expanded as needed to compensate for contour loss caused by previous processing.

[0100] In step 204, a color contour image is obtained based on the left image and the preprocessed left contour image.

[0101] In some embodiments, the pixel matrices corresponding to the left image and the preprocessed left contour image are multiplied by a dot product to obtain a color contour image. Thus, by multiplying the left image with its corresponding contour image, non-contour regions in the left image can be removed, while the contour regions are retained. This allows for subsequent disparity calculations using only the contour regions of the left image and the right image, eliminating the need for full-field calculations on the color image. This significantly saves computational resources and avoids wasted computational resources and slow processing speeds caused by invalid calculations.

[0102] In step 205, the disparity of the region containing the contour is calculated based on the color contour image and the right image.

[0103] Optionally, the disparity of the region containing the contour is obtained by calculating the disparity between the pixels corresponding to the contour position in the color contour image and the pixels corresponding to the same position in the right image.

[0104] In some embodiments, image correlation matching can be used to calculate the disparity between pixels corresponding to contour positions in a color contour image and pixels at the same positions in the right image. It should be noted that there are many image matching algorithms, such as Mean Absolute Difference (MAD), Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), and Normalized Cross-Correlation (NCC). This application does not specifically limit the image matching algorithm used.

[0105] In step 206, the disparity of the region containing the contour is compared and verified with the preprocessed left contour map and the preprocessed right contour map to generate a disparity map of the contour.

[0106] Optionally, the system iterates through the region containing each contour pixel (i.e., the position where the pixel value is 1), row by row, until the sum of the column number at the contour pixel position and the disparity at the corresponding position equals the column number of the right contour pixel. The valid disparity for a point is determined when the point is located on the contour in both the left and right images, and the difference between the horizontal coordinate of the point in the right image and the horizontal coordinate of the point in the left image equals the disparity calculated by the relevant matching.

[0107] As an example, such as Figure 3 The verification process shown can traverse each pixel in the first contour map, obtain the disparity value corresponding to the current pixel from the disparity of the region where the contour is located, add the coordinate value of the current pixel to the disparity value corresponding to the current pixel, and determine whether the coordinate value obtained after the addition calculation is on the contour position in the second contour map. If it is, the disparity corresponding to the current pixel is retained; otherwise, the disparity corresponding to the current pixel is deleted. This process continues until all pixels in the first contour map have been traversed, and the remaining disparity in the region where the contour is located is used as the disparity map of the contour.

[0108] For example, such as Figure 3 The verification process shown here, where the value of the contour position in the contour map is 1, means that the judgment condition L(i,j)==1and R(i+d,j)==1 means: the current pixel is on the contour position in both the left and right contour maps, and the difference between the horizontal coordinate of the current pixel in the right contour map and the horizontal coordinate in the left contour map is equal to the disparity corresponding to the current pixel. If this judgment condition is met, the disparity corresponding to the current pixel is retained, i.e., Disp(i,j)=d, where d is the disparity. If this judgment condition is not met, the disparity corresponding to the current pixel is set to null. Then, it is determined whether i is less than the total number of rows in the image and whether j is less than the total number of columns in the image. If so, j is incremented by 1, and the process returns to re-evaluate whether the next pixel meets the judgment condition. This process continues until all pixels in the first contour map have been traversed, and the remaining disparities in the contour area are used as the disparity map of the contour.

[0109] In step 207, a depth map of the contour is generated based on the disparity map of the contour.

[0110] In this embodiment of the application, when obtaining the disparity map of the contour, the focal length and baseline of the binocular camera can be determined. Using each disparity value, focal length and baseline in the disparity map, the depth information of each contour pixel in the contour map is calculated according to the following depth calculation formula (2).

[0111] Depth(i,j)=f*B / Disp(i,j) (2)

[0112] Where f is the focal length, B is the baseline, is the factory parameter of the binocular camera, Disp(i,j) is the disparity of pixel (i,j), and Depth(i,j) is the calculated depth of pixel (i,j).

[0113] According to the image processing method of this application embodiment, after acquiring left and right images with a binocular camera, image contour extraction is first performed. The target in the field of view usually has a large grayscale gradient with the background, allowing for the extraction of a very clear contour image. Then, the contour image of the left image is binarized to form a mask image, which serves as the disparity calculation area for the left image. The disparity of the region containing the contour is calculated and compared with the right contour image to obtain the disparity map of the contour. Finally, the depth map of the contour is calculated. This method does not require full-field calculation; it only calculates the depth map of the contour region, saving significant computational resources and improving computational speed. The disparity calculation in this method uses the binarized left contour image as a mask, effectively determining the calculation area and effectively excluding weak texture areas from the calculation range. Irrelevant weak texture areas will not be reflected in the contour image, effectively avoiding the influence of noisy weak texture areas on depth calculation.

[0114] Figure 4 This is a structural block diagram of an image processing apparatus according to an embodiment of this application. Figure 4 As shown, the image processing device 400 may include: a first acquisition module 401, a contour extraction module 402, a second acquisition module 403, a disparity map generation module 404, and a depth map generation module 405.

[0115] Specifically, the first acquisition module 401 is used to acquire a first color image and a second color image to be processed; wherein the first color image and the second color image are images from the same scene.

[0116] The contour extraction module 402 is used to extract the image contour from the first color image to obtain the first contour map.

[0117] The second acquisition module 403 is used to acquire a color contour image based on the first color image and the first contour image; wherein the color contour image is the image after removing non-contour regions from the first color image. As an example, the first color image and the second color image are the left and right images captured by the binocular camera, respectively. In some embodiments, the second acquisition module 403 performs a dot product operation on the pixel matrices corresponding to the first color image and the first contour image, respectively, in a matrix multiplication manner to obtain the color contour image.

[0118] Optionally, in some embodiments, the second acquisition module 403 preprocesses the first contour map and acquires a color contour image based on the first color image and the preprocessed first contour map.

[0119] The disparity map generation module 404 is used to calculate the disparity of the region where the contour is located based on the color contour image and the second color image, and to generate a disparity map of the contour based on the disparity of the region where the contour is located. In some embodiments, the disparity map generation module 404 can calculate the disparity of the region where the contour is located by comparing the pixels corresponding to the contour position in the color contour image with the pixels corresponding to the same position in the second color image.

[0120] In some embodiments, the contour extraction module 402 is further configured to extract image contours from the second color image to obtain a second contour map. In this embodiment, the disparity map generation module 404, after calculating the disparity of the region containing the contour based on the color contour image and the second color image, compares and verifies the disparity of the region containing the contour with the first and second contour maps to generate a disparity map of the contour.

[0121] As an example, the disparity map generation module 404 compares and verifies the disparity of the region containing the contour with the first contour map and the second contour map. The specific implementation of generating the disparity map of the contour is as follows: traverse each pixel in the first contour map, and obtain the disparity corresponding to the current pixel from the disparity of the region containing the contour; add the coordinate value of the current pixel to the disparity corresponding to the current pixel, and determine whether the coordinate value obtained after the addition calculation is on the contour position in the second contour map; if so, retain the disparity corresponding to the current pixel; otherwise, delete the disparity corresponding to the current pixel; until all pixels in the first contour map have been traversed, use the remaining disparity in the region containing the contour as the disparity map of the contour.

[0122] The depth map generation module 405 is used to generate a depth map of the contour based on the disparity map of the contour.

[0123] It should be noted that the foregoing explanation of the image processing method embodiments also applies to the image processing apparatus of this embodiment, and will not be repeated here.

[0124] According to the image processing apparatus of this application embodiment, a first color image and a second color image to be processed are acquired, and image contours are extracted from the first color image and the second color image respectively to obtain a first contour map and a second contour map. A color contour image is obtained based on the first color image and the first contour map, and the disparity of the region containing the contour is calculated based on the color contour image and the second color image. The disparity of the region containing the contour is compared and verified with the first contour map and the second contour map to generate a disparity map of the contour. A depth map of the contour is then generated based on the disparity map. Thus, by extracting contours from two images used for depth calculation, since the target in the field of view usually has a large grayscale gradient with the background, a very obvious contour image can be extracted. Therefore, full-field calculation is not required; only the depth map of the contour region is calculated, which can significantly save computational resources and improve calculation speed. Furthermore, by comparing and verifying the contour maps of the two images with the disparity of the region containing the contour, effective disparity can be extracted from the disparity of the region containing the contour. Therefore, the error obtained through the dual verification method of contour and disparity comparison can reduce the error of disparity calculation using a single calculation method, improving the accuracy and effectiveness of disparity calculation.

[0125] To implement the above embodiments, this application also proposes an electronic device.

[0126] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device 500 may include: a memory 501, a processor 502, and a computer program 503 stored in the memory 501 and executable on the processor 502. When the processor 502 executes the program 503, it implements the image processing method described in any of the above embodiments of this application.

[0127] To implement the above embodiments, this application also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image processing method described in any of the above embodiments of this application.

[0128] To implement the above embodiments, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the image processing method described in any of the above embodiments of this application.

[0129] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0130] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0131] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0132] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0133] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0134] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0135] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0136] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. An image processing method, characterized by, include: Acquire a first color image and a second color image to be processed; wherein the first color image and the second color image are images of the same scene from different angles; Image contour extraction is performed on the first color image to obtain a first contour map; Image contour extraction is performed on the second color image to obtain a second contour map; A color contour image is obtained based on the first color image and the first contour image; wherein, the color contour image is the image after removing the non-contour regions from the first color image; Calculate the disparity of the region containing the contour based on the color contour image and the second color image; Generating a disparity map of the contour based on the disparity of the region containing the contour includes: comparing and verifying the disparity of the region containing the contour with a first contour map and a second contour map to generate the disparity map of the contour; and A depth map of the contour is generated based on the disparity map of the contour.

2. The image processing method of claim 1, wherein, The first color image and the second color image are the first image and the second image captured by the binocular camera, respectively.

3. The image processing method of claim 1, wherein, The step of obtaining a color contour image based on the first color image and the first contour image includes: The pixel matrices corresponding to the first color image and the first contour image are multiplied by a dot product operation to obtain the color contour image.

4. The image processing method of any one of claims 1 to 3, characterized in that, Before obtaining the color contour image based on the first color image and the first contour map, the method further includes: The first contour image is preprocessed; wherein the preprocessing includes at least one or more of the following: denoising, binarization, and morphological dilation. The step of obtaining a color contour image based on the first color image and the first contour image includes: The color contour image is obtained based on the first color image and the preprocessed first contour image.

5. The image processing method according to any one of claims 1 to 3, characterized in that, The step of calculating the disparity of the region containing the contour based on the color contour image and the second color image includes: The disparity of the region containing the contour is obtained by calculating the disparity between the pixels corresponding to the contour position in the color contour image and the pixels corresponding to the same position in the second color image.

6. The image processing method according to claim 1, characterized in that, The step of comparing and verifying the disparity of the region containing the contour with the first contour map and the second contour map to generate a disparity map of the contour includes: Traverse each pixel in the first contour map and obtain the disparity corresponding to the current pixel from the disparity of the region where the contour is located. The coordinates of the current pixel are added to the disparity corresponding to the current pixel, and it is determined whether the coordinates obtained after the addition are on the contour position in the second contour map. If so, retain the disparity corresponding to the current pixel; otherwise, delete the disparity corresponding to the current pixel. Until all pixels in the first contour map have been traversed, the remaining disparity in the region where the contour is located is used as the disparity map of the contour.

7. An image processing apparatus, characterized in that, include: The first acquisition module is used to acquire a first color image and a second color image to be processed; wherein the first color image and the second color image are images of the same scene from different angles; The contour extraction module is used to extract the image contour from the first color image to obtain a first contour map. The contour extraction module is also used to extract the image contour from the second color image to obtain a second contour map; The second acquisition module is used to acquire a color contour image based on the first color image and the first contour image; wherein the color contour image is the image after removing non-contour regions from the first color image; A disparity map generation module is used to calculate the disparity of the region containing the contour based on the color contour image and the second color image, and to generate a disparity map of the contour based on the disparity of the region containing the contour; and A depth map generation module is used to generate a depth map of the contour based on the disparity map of the contour. Specifically, the disparity map generation module is used to: compare and verify the disparity of the region where the contour is located with the first contour map and the second contour map, and generate a disparity map of the contour.

8. The image processing apparatus according to claim 7, characterized in that, The disparity map generation module is specifically used for: Traverse each pixel in the first contour map and obtain the disparity corresponding to the current pixel from the disparity of the region where the contour is located. The coordinates of the current pixel are added to the disparity corresponding to the current pixel, and it is determined whether the coordinates obtained after the addition are on the contour position in the second contour map. If so, retain the disparity corresponding to the current pixel; otherwise, delete the disparity corresponding to the current pixel. Until all pixels in the first contour map have been traversed, the remaining disparity in the region where the contour is located is used as the disparity map of the contour.

9. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the image processing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the image processing method as described in any one of claims 1 to 6.

Images